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Preface 



We propose a fundamentally new national data system for elementary and secondary 
education. Our proposal differs in structure and content from present education data-collection 
activities of the federal government, the states, and local education agencies. Only a fundamentally 
new system can produce essential data for the nation and the states that are correct, accurate, 
precise, timely, comparable, and useful. Our proposal may appear costly, demanding, and 
complex, and will require a long-term commitment from the federal government and the states, but 
we believe there is no alternative. We have reached this conclusion following an intensive and 
careful review of recent changes in education, consequent new demands for information, the 
capabilities of present education data systems, the appropriate federal role in a national data system, 
and the fundamental characteristics of a responsive data system. 

Our proposal is fundamentally different because: 

1. the content of the data system is derived systematically; it is based on a clear 
conception of what education is, and how education operates in the United States; 

2. the structure of the data system provides linked data elements, data files, and data 
records; data are collected about and maintained for individual students, teachers, 
schools, and school systems; 

3. data provided by the system will enable principal policy-makers and other 
information users to understand the context, processes and outcome? of schooling in 
the United States; 

Further our proposed data system is characterized by 

1. state representative samples of public and non-public schools for all states; 

2. a necessarily high level of state and federal cooperation; 

3. a coordinated set of federal and state collection efforts; 

4. a data base which will provide new data to policy makers at all levels of government 
as well as data for education research. 

The proposed program will provide all data presently collected through all current NCES 
projects concerned with data on elementary and secondary education. This includes the 
longitudinal survey program, the National Assessment of Educational Progress, the Common Core 
of Data, the Vocational Educational Data System, and the several discrete sample surveys of 
teachers, private schools, etc. 

Our report was contracted by the U.S. Department of Education's Center for Statistics as a 
"ten-year plan" of data collection to satisfy the statements made by writers submitting papers to the 
Center's Redesign Project and appearing in the Synthesis of Invited Papers. It should be 



ERJC 



i 

5 



considered a companion volume to the Synthesis of Invited Papers. 

Some Questions and Answers About the Report . An earlier draft of our proposal was 
circulated to key education policy-makers outside and within the federal government. They 
responded with a set of pertinent and provocative questions that demand thoughtful consideration. 
Many of these questions are answered in various sections of our report. A synopsis of these 
questions, and a guide to report sections that provide answers follows: 

Q. Why should the Department of Education want to install a totally new national 

data-collection system, instead of fixing up the one that exists? 
A. The answer is twofold. First, the current data system is flawed in fundamental ways; it does 
not produce traditional statistics that are useful, accurate, comparable, and timely, according to the 
authors of many of the papers that underlie the Synthesis Report (see Part I, Chapter 2). Second, 
the current data system does not, and cannot, provide the kinds of information needed to 
understand the context, processes, and outcomes of schooling in the United States. These kinds of 
information are now being demanded by policy-makers at all levels of government as well as by the 
general citizenry, (see Part I, Chapter 1). 

Q: How do you know that the system you are proposing will really work? 
A: To * modest extent, linked data records for students, teachers, and schools have been used in 
the past in such Center for Statistics projects as the national longitudinal studies and NAEP. The 
proposed national data system will capitalize on the experience gained in such projects. In addition, 
for those elements of the proposed system that are novel, design, development, and 
implementation procedures include 1) a program of intensive research and development of novel 
system components, and 2) a period of testing and verification of all newly-developed system 
components prior to their operational use (see Part II, Chapter 5, Section G). 

Q: Isn't there a whole lot of data burden involved from introducing a system additional to the 

existing state management information systems? 
A: Data burden should be considered from the perspective of state education agencies and local 
education agencies. For a state education agency, the degree of added data burden will depend 
upon the state's decision either to continue its current data systems, separate from the federal 
system, or to integrate its data systems with the national system. In the former case, there will be 
little added data burden for the state agency. In the latter case, the state would likely have to modify 
its present data systems (and might thereby incur substantial development costs); however, it is not 
clear that the state would have to provide any more data to the Federal Center for Statistics than are 
presently required. The added burden imposed on local education agencies would also depend on a 
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state's decision to integrate its data systems with the national system. If a state 
education agency maintained its data systems, separate from the federal system, only the sampled 
local agencies within that state would be required to provide data to the federal system. Added data 
burden for those sampled LEA's might be substantial. If a state education agency integrated its data 
systems with the national system, all local agencies within the state might be required to provide 
additional data for the state's system, and possibly for the national system. The magnitude of 
added data burden for local education agencies would depend on the content of the data systems 
operated by their states, as well as the content of the federal component of the national data system 
(see Part n, Chapter 5, Section G). 

Q: Aren't you really talking about a lot of data for researchers instead of statisticians? 
A: No. The proposed national data system is designed to provide essential information for 
policy-makers in all branches and at various levels of government as well as the new constituencies 
identified in our report (see Part I, Chapters 1 and 4; Part II, Chapter 5, Section B). It might also 
he the case that data collected for essential policy purposes will be of interest to a wide variety of 
educational researchers, but serving their needs is noj the principal function of the proposed data 
system, nor has the system been designed to serve maximally the interests of the research 
community. 

Q: What would it cost to develop and implement the kind of data-collection system you are 
describing? 

A. The types of costs and difTibution of costs likely to be incurred in developing and 
maintaining the proposed national data system are enumerated in our report (see Part n, Chapter 5, 
Section F). However, we recommend that carefully-derived estimates of the costs of developing 
and maintaining the proposed national data system be secured from professional survey research 
agencies in the public and private sectors. 

Q: How long would it take to get data from the kind of system you are describing? 
A: The proposed system would provide some data during its first six- month phase and provide 
data reproducing currently available tables shortly thereafter (see Part n, Chapter 5, Section G). 
However, we anticipate that the proposed system would not be fully operational with respect to 
newly proposed data claims before July, 1991. We recognize that the Federal Center for Statistics 
must meet its ongoing Congressionally-mandated responsibilities to pioduce information on the 
status and condition of education in the United States (see Part I, Chapter 3). While the new 
national data system is underdevelopment, all of these responsibilines would be met through a 
combination of information provided by the new system and appropriate current projects of the 
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Federal Center for Statistics (see Pan II, Chapter 5, Section G). 



O: Why didn't you identify the data items to be included in the system? 
A: The structure of the proposed national data system, as well as specific categories and 
subcategories of data have been identified in our report (see Part I, Chapter 4; Pat II, Chapter 5, 
Section D). Specific data items must be joindy identified by the Center for Statistics and 
representatives of the constituencies the system is designed to serve, if the system is to effectively 
meet the needs of these data users. We have proposed a strategy for securing, on a collaborative 
basis, the judgments of groups of information users on the data elements that would best meet their 
needs (see Pan II, Chapter 5, Section G). 



ERLC 



4 

8 



Summary 



The most important fact about elementary and secondary schooling in the United States 
today is that there is almost universal dissatisfaction with its quality. This dissatisfaction, in turn, 
has begun to force major changes in the educational system; and these changes are taking place 
within new contexts for making educational decisions. No longer do these decisions fall within the 
exclusive purview of local school boards and local school administrators. Parents are exploring 
new alternatives for the education of their children. In the ongoing educational reform, state 
officials and public bodies are expanding the range of their actions in attempts to improve the 
quality of schooling. And information about the quality of education-about the quality of schools, 
school districts, and state educational systems-has become a priority concern for the increasing 
array of new policy acton now involved in making decisions that affect the quality of the education 
being received by the Nation's children and youth. 

In the past, the primary information users were the education professionals responsible for 
conducting the educational process. These decision makers participated in local information 
systems focused around day-to-day decisions made by teachers and administrators. Such 
information systems generally were limited to providing basic data on the local level about pupils, 
personnel, educational services, and finances. Now, however, we are finding that information 
must be conveyed beyond the boundaries of the local educational agency. Parents need information 
about the character and quality of the the educational alternatives available to their children. The 
citizenry— local, state, and national- requires and is demanding information about the quality of its 
schools. The business community-at local, state, and national levels-is calling for systems that 
will provide quality assurance information on the schools. State officials-governors, legislators, 
state school board members, officers of state education agencies -require more comprehensive and 
accurate information on the quality of the schooling being conducted within their purview. Federal 
officials-from the President through the Congress to the agency adminstrators-are seeking 
information on the general condition of American education, as well as on the effectiveness of 
federal education policies and programs. 

Fortunately, in our view, the U.S. Education Department and The Office of Educationa' 
Research and Improvement view the current climate of reform as an opportunity "to seize the day," 
to develop the new data acquisition programs that will provide the public the information that it 
needs and wants, the policy makers the information they need to judge the efficacy of the reform 
efforts, and the educational community the information it needs to monitor these efforts over time. 
In our view, the Elementary/Secondary Education Redesign Project comes at a most opportune 
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time, a time when our long- standing faith in the American public school system is being seriously 
challenged, a time when serious- minded reformers are proposing substantive changes in traditional 
modes of school operation, a time when the education community has begun to fashion exciting 
and promising responses to these calls for reform, and a time when an entirely new configuration 
of education information and data needs is presenting itself. 

The Mission of the Center for Statistics 

A unified and coherent program for acquiring, analyzing and disseminating information on 
the condition of education throughout the United States cannot exist without a centralized 
administrative unit within the federal government that has, as its sole mission, accomplishment of 
those goals. On March 1, 1867, the Congress acknowledged the need for an agency in the 
Executive Branch that would meet the nation's needs for information on education. From its very 
beginning, the central purpose and focus of the Department of Education was the collection, 
analysis, and reporting of information on the condition and progress of education, for the dual 
purposes of helping states and local school systems improve their effectiveness and informing the 
Congress on the general status of and returns to the federal investment in education. The role of 
The Center for Statistics in fulfilling this mission must now be judged against existing and new data 
needs. For example, many authors of papers for the NCES Redesign Project cited inadequacies in 
the scope and coverage of data presently collected, and the inability of the present NCES projects to 
provide information that can be used to compare the condition and progress of education in the 
various states. 

With the exception of the Common Core of Data, most of the data-collection activities of 
The Center for Statistics support estimates of parameters only for nationwide, or occasionally for 
regional, populations. Yet education is an activity that is constitutionally reserved to the states, 
and, as noted by many of the contributors to the NCES Redesign Project, the vast majority of 
important education policies originate at the state level of government. The need for education 
statistics that support comparisons across states has been strongly voiced. 

Many contributors to the Redesign Project view the 1974 statement of the Center's mission 
(defined in the General Education Provisions Act, as cited above) as inadequate to the need for such 
a center, and, as a step backward from the original charter of the Department of Education. 
Although the 1974 mission statement defines a specific set of responsibilites for The Center for 
Statistics, it does not empower a unitary federal agency with sole authority and responsibility for 
informing the nation on that sector of society labeled "education." As a result, NCES does not do 
enough and other agencies of the federal government, both within and outside the Department of 
Education, do too much in their quest for data on schooling and education in the United States. 
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Unncessary duplication, lack of coordination, and excessive responded burden are well 
documented in the papers prepared for the Redesign Project. 

The mission we would propose for the Center would make it the federal agency with 
authority and responsibility for collection of data concerning education in the United States. Other 
agencies with specific needs for regulatory data on education (such as the Off ce for Civil Rights, 
other agencies within the Department of Education, the Department of Labor, the Department of 
Agriculture and the Department of Defense) might collect such data, provided their activities were 
coordinated by the Center, and only in situations where it has been unable to meet the^r 
programmatic needs for such data. 

Our position on an appropriate mission for The Center for Statistics is consistent in spirit 
with that advanced by the Council of Chief State School Officers (1985): 

"We strongly urge that the function [of NCES] be a true statistical center that assumes the 
major responsibility for coordination of the collection, assembly, analysis and dissemination 
for that sector of society under its purview, namely education. 

At the core of new requirements is the need for an integrated program of data collection, 
analysis, and reporting, in contrast to the largely unarticulated set of data collection projects that 
presently operates at all levels of the education enterprise. Although major factors that influence the 
operation of the nation's education systems are inextricably linked; e.g., changes in levels of 
educational resources produce changes in the availability and quality of education personnel, and 
these, in turn, produce changes in educational offerings and sendees. Present data-collection 
projects are inadequate to describe and characterize such linkages. In many cases, the structure and 
articulation of interrelationships among components of the nation's education systems muSi be 
deduced from unexamined inferences or from incomplete and inappropriate empirical findings. No 
education data system or program adequately characterizes the whole of the education enterprise. 
Although relationships between educational resources, expenditures for personnel, and the quality 
of personnel are known in part, there is no integrated educational data system to document the 
extensiveness and operation of such relationships, either for the nation as a whole or for individual 
states. 

Criteria for a New Educational Information System 

In designing a new data system, one needs to answer several critical questions. These 
include: 

1. What information should be collected? 
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This question addresses not only the "contents" of individual information elements, 
but also their form, i.e., fundamental linkages between elements that allow or 
prohibit their use for specific purposes. 

2. How should the information be collected? 

This issue encompasses not only the methods of data collection, but also the 
categories of persons and administrative records which will provide the data. 
Sample design, timing of collection, and provision of standards of data 
quality-including mechanisms for assessment and con ol-are also key issues. 

3. How should it be made available for use? 

This latter question breaks down to: Who should receive what information, in what 
form when, and at what cost? 

These issues address data transmission processes among those reponsible for collecting the 
data and maintaining the system as well as information flows linking external users into the system. 
As such they include specification of records to be transmitted, timing and frequency of 
transmission, aggregations and analyses to be performed and reported, regulation of 
access— including timing of data releases, provision of privacy and security constraints, availabiity 
of micro-versus aggregate records, the costs of access and who should bear them. 

An educational system is an organization which converts resources into educational 
services for pupils. From our perspective, one can specify public education as a system at the level 
of class, school, district, or state. These form a nested set of educational systems, with varying 
and changing responsibilities for governance and policy formation. Private educational systems 
typically have fewer organizational levels. 

Ultimately, the success of an educational system-regardless of organizational level-is 
predicated on its outcomes. As a society, we intend the system to help prepare individuals for 
work, for political participation, and for family life. To the extent that education does not play its 
role in preparing students effectively, we desire to improve it Because of the central role of 
outcomes in evaluating the success of our educational systems, greatly increased efforts have been 
made recently to improve amount and character of information about pupils' achievements. The 
intent of these changes--at local, state, and national levels-has been to assess the quality of our 
educational system and to bring about improvements in them. 

A Conceptual Model 

In order to apprehend educative processes, we must rely upon a conceptual model. This 
model may be simple or complex and it may be implicit or explicit, but its existence is a prerequisite 
to any understanding of the effectiveness and quality of schooling. Our conceptual 
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framework-presented below-for c scribing an educational system focuses on the school because 
it is at the level of the school that educational activities take place and that pupils participate in them. 
Fundamentally, schools and tlie communities they serve differ in several important ways: 
Family and Community Environment, The families and communities served by different 
schools differ in significant ways. They differ in die resources available in the homes of the pupils 
for support of their schooling and they differ in types and levels of aspirations parents have for 
their children. The family composition of the community affects the attitudes, values, and goals of 
a pupil's peers. All of these form the context within which schools can educate their pupils. 

Educative Difficulty . Schools are faced with differences in levels and types of educative 
difficulties with which their pupils present them. Some present handicaps or limited proficiencies 
in English. Others come with limited levels of prior learning. Thus, pupils who enroll in some 
schools enter with cognitive accomplishments and capabilities, motivations, and out-of-school 
environments and resources which make educative efforts easier and less complex than those in 
other schools. 

Resources . Schools have available to them different levels of monetary resources and 
different amounts and k ; nds of non-monetary resources, such as volunteer time, donated supplies 
and equipment. 

Ti^ese resources are exchanged, allocated, and configured as a teaching staff, facilities, 
educational materials. 

Goals . Schools aspire to distinct ve goals. For example, some public secondary schools 
design their entire curriculum around post-secondary career paths which primarily begin in 
selective colleges and universities , while other schools, e.g., "vocatio ial" ones, may focus their 
whole program around immediate jcj enny to skilled and semi-skilled occupations. 

Process . Schools offer educative experiences for which they require or encourage pupils' 
participation. These include work exp^ience, homework, and extra-curricular activities as well as 
in-class experiences. Schools also structure these experiences with different standards. These 
standards influence the pursuit of goals with different expectations for performance, differing time 
allowances for accomplishment, and differing criteria for selection into subsequent experiences. 

Schools also differ in the types and amounts of participation of their pupils in these 
educative experiences as well as in the range of experiences made available. These variations 
include differences in selection, participation, and completion of educational programs, course 
work, aixi homework as well as differential school attendance. 

Outcomes . All through the schooling process, to the conclusion of secondary schooling 
and beyond, schools differ greatly in the goal-relevant accomplishments and achievement of their 
pupils. These include cognitive capabilities, credentials, and career and life paths generally. 

Figure 1 in Chapter 4 displays such a conceptual framework. It focuses on the schooling 
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process, distinguishing teaching activities from pupils' exposure and participation in the resulting 
educative activities. And it traces these aspects of the process to their origins: prior and 
contemporaneous characterises of pupils, community and family expectations, curricuiar goais, 
and resouices, as well as Linking them to their consequences. 

The consumers of information about educational systems include parents concerned about 
the education of their children, citizens worried about the quality and efficiency of the education 
their tax dollars finance, professional educators making decisions about programs and pupils, arid 
public officials desiring to design laws, requirements, and resource allocations which will 
effectively improve education. All of these consumers are concerned that the information which 
reaches them be relevant and useful to their needj>, be timely, and be accurate. 

Common to them all are concerns about quality and effectivenss. It is this information 
which is most desired in the public debate over education. Parents want to know about the quality 
of education their children receive and about the qualities of the educational alternatives available to 
them. Citizens and public officials wish valid assessments of efficiency to know that resource 
allocations are wisely made and carried through desired outcomes. 

Resource flows are important information for public officials in making determinations of 
how much and how to allocate resources. Federal officials have special concern for how federal 
resources are channeled to pupils and the impact of these resources on pupils with specific 
characteristics. State offic i, in fulfilling their responsibilities, have been modifying state 
educational systems in ways that require comprehensive information about participation in 
programs, courses, and other services, standards of performance and actual outcomes. Local 
officials are newly concerned that they are effectively monitoring service delivery, participation and 
achievement. 

An effe' jvely integrated system-incorporating the microdata and records necessary to 
produce these new types of information-is needed by all concerned parties. The benefits of a 
cohesive system of this type producing national and state comparable data would be far reaching. 
Not only would the majority of consumers of educational information be provided with relevant, 
integrated, timely, and accurate information at these two levels, but the establishment of such a 
system would produce similar changes in district-level information systems. This, in turn, would 
increase the comprehensiveness and comparability of the information about education taking place 
in local communities. Thus, the national information system, as it is established at state and 
national levels will introduce cohesion in the total system. 
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What is a National Educational Information Svstem? 



Wc have discussed at length the need for an integrated national educational information 
system built around a comprehensive conceptual model of the schooling process. In the 
development of such a system, three questions need to be addressed. What sort of data base is 
called for ? What processes will be used to get information into the data base? What processes will 
be used to get information out of the data base? We offer general answers to these three questions 
and, thereby, describe in broad terms our view of what a National Educational Information System 
must be. 

The Data Base . In order to meet the information needs of the broad array of local, state, and 
national educational decision makers identified in previous chapters, the data base must be 
structured to provide information on all aspects of the schooling process as described in our 
conceptual model. This means that the data base must be comprehensive; put simply, it must be 
adequate in scope and coverage; it must contain accurate, appropriate, and timely information on (1) 
the school setting, (2) the schooling process itself, and (3) the outcomes of schooling. 

Data on the school setting must include information on the environmental factors that 
impinge on the school, such as community and family characteristics and expectations. School 
setting data also must provide information on financial revenues as well as other incoming 
non-monetary resources available to the school. The data base also must include information on the 
educative difficulties which face the school, such as pupils' capabilities, motivations, handicaps, 
English language facility, and out-of-school supports. 

Data on the schooling process must include a broad range of information. The data base 
must provide information on the educative goals of the school, its objectives, and curriculum. It 
must provide information on allocated resources-facilities, staff, equipment, materials, and other 
non-monetary resources made available for school use. It must provide information on educational 
pursuits, that is, curricular offerings, standards, teaching-and school-related activities. Equally 
important, the data base must provide information on the extent of pupil and parent participation in 
the process of schooling. 

Finally, the data base for a national eductional information s ystem must include information 
on the outcomes of schooling, including pupil achievement data as well as information on such 
outcomes as high school graduation, drop-outs, political participation, employment, and 
post-secondary matriculation. 

A second major requirement of the data base, in addition to being comprehensive, is that it 
be integrated, that is, that its data elements, files, and records be linked to one another. The user 
must be able to ask and have answered questions about the relationship among background 
characteristics, the schooling process itself, and the outcomes of the process. The data base must 



ERLC 



li 

15 



be able to provide information to answer such questions as, "What dollars buy what services for 
which students with what results?" Or. "What programs staffed by what types of teachers are 
effective for pupils with particular educative difficulties, at what costs?" Only if the data base is so 
stm ;tured as to allow relevant linkages among it element?, files, and records, will the requirement 
for an integrated educational information system be met. 

Getting Information into the Data Base . Tne dual requirements for a comprehensive and an 
integrated system demand, in turn, that data be collected in micro record form, as opposed to 
macrorecord or aggregated form. We define a micro record as a data set concerning an individual 
entity rather than a data set on a collection or aggregate of individual entities. A micro record can 
be dealt with as an individual datum or aggregated; for example, individual micro records on 
pupils can be aggregated to the school level. A macro record on the other hand, generally cannot be 
disaggregated. More importantly ; the micro record permits of linkages with other micro records; 
for example, micro records on individual pupils can be linked with micro records on individual 
teachers and, in turn, with micro records on specific curricular offerings in which the teachers and 
pupils are participating. The micro record format, through its linkage capability, permits the 
information user to ask questions about relationships among the sets that make up the data base. 
Thus, a major requirement in designing a process for getting information into the data base we have 
described above is that the information be collected and stored in micro record format 

Getting Information Out of the Data Base . We have identified the major requirements that 
must be met in establishing the data base and for getting information into the data base. A third 
question remains, namely, wh -t processes will be used for getting information out of the data oase? 

First, a National Educational Information System must be able to deliver information of a 
comprehensive and integrated nature on the schooling process in the Nation as a whole, that is, it 
must be capable of delivering information that is nationally representative. It must be able to report 
on the status and progress of elementary and secondary schooling in the United States. It also must 
be able to deliver information on sub-national or regional systems and populations. In addition, we 
have taken as a given tha; the system must be capable of producing information that can be used to 
compare the condition and progress of education in the various states; in short, the system must be 
capable of delivering information that is representative of each of the fifty states. 

While such requirements dictate attention to how information gets into the data base, e.g., 
the sampling designs which will be employed, they also dictate-along with the previously 
identified requirements of comprehensivenss, integration, and micro record formats-what types of 
reports must be available to users of the system. Users, with the possible exception of researchers, 
generally will not be interested in micro records per se but rather reports developed from the 
processing-e.g., tabulation, aggregation, and analyses-of micro records . Thus, while micro 
records represent the form in which information flows into the data base, reports based on 
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processing of the micro records represent the form in which information flows out of the data 
base. Yet, a simple proliferation of reports will not meet the needs of the broad array of local, 
state, and national decision makers which we have identified in previous chapters. A national 
educational information system must be capable of carefully tailoring its reporting formats and 
mechanisms if it is to serve the particular needs of this broad array of decision makers. Certain 
decision makers, for example Governors, have needs for only certain kinds of information and not 
for other kinds; the system must be capable of meeting these needs. In short, the system must be 
capable of screening and matching its reporting formats with the needs of particular users. In 
addition to questions of content, the screening and matching require attention to establishing the 
mechanism necessary to actually get the reports to decision makers and decision makers to the 
reports and, La the case of researchers, to the relevant portions of the data base itself. 

Finally, the process for getting information out of the system has to pay serious attention to 
timing. Unless the information is available when needed, the content and form of the reporting 
mechanism makes little difference. Timing involves setting priorities for reporting different sets of 
information to different users, as well as priorities for providing different users access to different 
sets of information. 

In summary, a national educational information system must be capable of delivering 
periouiw and differentiated reports on the status and progress of schooling to a broad array of local, 
state, and national decision makers, as well as making available to different users, including 
researchers, special reports on and public use samples relevant to particular aspects of elementary 
and secondary schooling in the United States and in the several states. 

A Mechanism for Cooperative Engagement 

To develop the detailed design for the new National Data System, the Center, working 
through the Chief State School Officers, should establish a consortium of all states and develop an 
agenda for identifying specific information elements and data elements required for the system. 
The Center should also appoint a number of other members to the consortium, including 
representatives of local education agencies, academia, and others concerned with information about 
the educational system. The consortium should also have a say about the method in which the data 
base is organized and what data, in what form, would become available. 

It would be foolish to believe that a body representing this large a constituency could do the 
detailed planning required for this effort. There wouM be an obvious need to develop working 
groups to address specific issues. For example, several states are in the process of developing state 
lev.; integrated information systems; each is designed to provide the specific data needed for state 
purposes. In order to foster the development of compatible systems to produce comparable data, 
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the Center should attempt to organize a working group of the consortium consisting of states 
already developing such systems, along with other states interested in similar development Since 
in general the systems would be integrated, it would be essential for iocal systems to be 
represented This would facilitate the exchange of information among the states and the 
development of alternative models which could feed the national data base. 

Although the Center would have the responsibility for staffing the consortium and 
establishing working groups for the various technical issues which will have to be addressed 
during the development of the system, the total input to the Center should more than compensate 
for the cost of staffing. 

Recommendations 

1 . The Center for Statistics should create a national data base of micro records for pupils, 
educational personnel, districts and schools, both public and non-public. 

2. The national data base should 

a. incorporate relational linkages among files, 

b. cover family and community environment, educative difficulties of pupils, 
resources, goals, schooling process, and outcomes, 

c. accurately represent the nation as a whole and the individual state educational systems 
which compose it and, therefore, 

d. permit accurate comparisons of state educational systems. 

3. This data base should form part of a comprehensive national education information system 
incorporating; 

a. options for state participation in data collection, and 

b. a comprehensive system of data access and dissemination. 

4. The system should be phased with a planned schedule of development and partial 
implementation leading to full implementation within five years. 
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Part I. Context, Needs and Rationale 
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Chapter 1 

Changes in Education and Changing 
Demands for Information 



The most important fact about elementary and secondary schooling in the United States today 
is that there is almost universal dissatisfaction with its quality. This dissatisfaction, in turn, has 
begun to force major changes in the educational system; and these changes are taking place within 
new contexts for making educational decisions. No longer do these decisions fall within the 
exclusive purview of local school boards and local school administrators. Parents are exploring 
new alternatives for the education of their children. State officials and public bodies are expanding 
the range of their actions in attempts to improve the quality of schooling. And information about 
the quality of education—about the quality of schools, school district, and state educational 
systems-has become a priority concern for the increasing array of new policy actors now involved 
in making decisions that affect the quality of the education being received by the Nation's children 
and youth. 

A. Rising Public Concerns About Educational Quality 

Dissatisfaction with American education has been building for a number of years. During at 
least the last ten to fifteen years, American education has been experiencing a growing period of 
unrest, of harsh criticism of its practices, of general skepticism about its ability to serve the Nation. 
Reports of declines in SAT scores, of unfavorable comparison of achievement levels among 
American youngsters and their counterparts in Japan and European industrial nations, of the lack of 
teachers adequately prepared to teach math and science (or other subjects for that matter), of high 
school graduates who are functional illiterates, all have been producing a rising sense of alarm 
among the American public. The 1970's and early 1980's saw a growing feeling on the part of the 
citizenry that many of our Nation's young people were not being properly prepared for entry either 
into colleges and universities or into the American work force. These sentiments perhaps were 
expressed most visibly in the recent report of the National Commission on Excellence which 
contends that "...the educational foundations of our society are being eroded by a rising tide of 
mediocrity." 

But the Commission's report, in truth, made no new discoveries. Well before the issuance 
o f A Nation at Risk concerns about the serious problems of the schools had been receiving 
increasing attention from a number of writers: 
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Looking back, we can now see that by the late 1970's a critical mass of writers, 
intellectuals, and academics-few of them deeply into the educational 
establishment-were beginning ro be heard on the failures of the schools: Diane 
Ravitch, Chester Finn, Dennis Doyle, Tommy Tomlinson. At the very end of that 
decade, one was aware that a number of large studies of high schools were under 
way-by James Coleman, Gerald Grant, John Goodlad, Ernest Boyer, Theodore 
Sizer-all inititated by mounting uneasiness about the condition of secondary 
education (Adelson 1985). 



In the early 1970's, an air of discontent about the quality of secondary schooling in this country led 
the Charles Kettering Foundation to establish a National Commission on 'he Reform of Secondary 
Education. The Commission's charge was to: 

...make a comprehensive examination of secondary education and provide the 
American public with a clear, factual picture of their secondary schools, indicating 
where and how they can be altered to better serve the Nation's young people 
(National Commisison on the Reform of Secondary Education 1973). 



In the middle 1970's, nationally syndicated columnist Neal R. Pierce, writing in the Washington 
Post captured the growing discontent among the Nation's governors on matters of public 
education. He noted, for example, that Richard Snelling, the Governor of Vermont, was 
suggesting that the public schools were performing their roles so poorly, "that President Carter 
should call a Constitutional Convention on education in America." In Pierce's words: 

The hard facts are that in schools from coast to coast, verbal and mathematic 
Scholastic Aptitude Test scores have fallen steadily since 1963-almost without 
regard to whether the school system is poor or rich, center city, suburban or rural. 
Reputable surveys have shown that 12 of every 100 17-year old high school 
students are functionally illiterate, that scarcely 50 percent know that each state has 
two senators or that the president can't appoint members of Congiess...No one 
believes the schools' problems will be quickly or easily solved. But across the 
nation, the ferment for change is growing rapidly (1977). 



The growing ferment that Pierce described produced a number of common themes-themes 
expressed not only by governors, but by state legislators, state education officials, education 
interest groups, and increasingly by parents and coalitions of parents and other concerned citizens. 
Chief among these themes was a call for a return to the basics, a demand for stricter school 
discipline, a demand for minimal competency testing, and a growing resentment against an 
educational establishment that constantly sought more funds but stubbornly resisted external 
monitoring. 

Other journalists, in addition to Pierce, were increasingly writing about the problems faced 
by the Nation's schools; and many were entering the growing search for "effective schools." 
Robert Benjamin, one of many writers supported under the Ford Fellows in Educational 
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Journalism Program of the Washington, D. C-based Institute for Educational Leadership 
(Brundage 1977) chronicled his coast-to-coast search for effective schools it: . 981 book, Making 
Schools Work . In his introduction, Benjamin captures both the sense of increasing public alarm 
over the plight of the schools--particularly the urban schools, and the importance of broadening the 
stakes beyond those of the professional educator: 

The quest was hopeful: What makes schools work well? But it was set against a 
discouraging backdrop: the persistent failure of this nation's public schools to 
educate low income students by even minimal standards. 

It was a reporter's journey, rather that a professional educator's. It was undertaken 
with the belief that the benefits of this viewpoint outweigh its limitations, that all of 
us have a clear stake in shaping the solutions to what may prove to be the most 
challenging problem faring America's cities in the 1980's (1931). 

Concerns about the quality of the Nation's schools also increasingly were being expressed 
by the business community which say, in the failures of the schools, serious problems for the 
national economy and grave threats to America's traditionally favored position in the world of 
international competition. Calls for deepened business community involvement in the schools were 
forthcoming from a number of quarters. The New York Stock Exchange, in its report, People and 
Productivity: A Challenge to Corporate America , advocated a strong effort to raise business' 
awareness about their stake in the problems the schools were facing: "We must understand that 
schooling is a long-term investment in human capital, and that productivity suffers when that 
investment is neglected" (1982). That the business community did become aware of the problems 
and did move to become involved is evidenced in part in one of several major education reform 
reports released during 1983, Action for Excellence , the report of the Task Force on Education for 
Economic Growth, whose membership consisted primarily of governors and business leaders from 
some of America's major corporations, rather than professional educators. The Task Force report 
argues that: 

Technological change and global competition make ii imperative to equip students in 
public schools with skills that go beyond the "basics."...Mobilizing the education 
system to teach new skills, so that new generations reach the high general level of 
education on which sustained economic growth depends, will require new 
partnerships among all those who have a stake in education and economic growth. 

Thus, by the middle of the decade of the 1980's, public education had become an object of 
great concern to a wide array of Americans-to parents, to other citizens, to state education officials, 
to governors, to state legislators, to broad-based interest groups, to the business community, and to 
a host of other Americans who now began to see themselves as increasingly important stakeholders 
in the Nation's schools. 
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Fortunately, in our view, the U.S. Education Department and the Office of Educational 
Research and Improvement (OERI) view the current climate of reform as an opportunity, "to sieze 
the day," to develop the new data acquisition programs that will provide the public the information 
that it needs and wants, the policy makers the information they need to judge the efficacy of the 
reform efforts, and the educational community the information it needs to monitor these efforts over 
time. In our view, the Elementary/Secondary Education Redesign Project comes at a most 
opportune time, a time when our long-standing faith in the American public school system is being 
seriously challenged, a time when serious minded reformers are proposing substantive changes in 
traditional modes of school operation, at time when the education community has begun to fashion 
exciting and promising responses to these calls for reform, and a time when an entirely new 
configuration of education information and data needs is presenting itself. 

B. The Changing Nature of Educational Decision making 

In the opening paragraph, we noted that growing dissatisfaction with the quality of schooling 
has begun to force major changes in the educational system, including major changes in the 
contexts in which educational decisions are being made as well as major changes in the cast of 
educational decision makers. These changes forced an opening up of the decision making process 
and resulted in almost totally erasing public education 'a traditional identity as a separable and 
special governmental operation. These changes resulted in pulling educational issues into the 
political mainstream; in opening up the system to parents, to the general public, to general 
government, and to special interests; and in forcing professional educators to integrate diverse 
segments of the community into the decision making and policy making processes of education. 

American public education, for a long time, enjoyed an identity as a separable and special 
governmental operation. It was viewed by many as almost sacred. Its basic liturgy held that 
education was a unique function of government; that it must have its own separate and 
politically independent structure; that it should be uninvolved in "politics," indeed, that it should be 
divorced from politics; that professional educators alone should be involved in major decisions 
about education; that professional unity among educators was the norm. In effect. America's public 
schools were characterized by what political scientists would call "closed system politics." Schools 
were politically isolated, with tight boundaries, and neither elected politicians nor the general public 
nor parents had much success in cracking open the system and gaining access. 

The state's policy role was minimal. Most states left most educational matters to local 
discretion. But those times have passed— perhaps forever. In state after state, public education has 
lost its privileged place as a unique governmental function. It is, more and more, beginning to be 
seen as just one of several human services competing for the public dollar. 
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At the local level, the Nation has witnessed and continues to witness a redistribution of the 
political influence previously held almost exclusively by professional educators and primarily by 
local school superintendents and local school boards. Parents, teachers, minority groups, students 
and others have successfully pressed their ca:es. They have gained access to and have assumed 
significant roles in the educational decision making process. And, ate, this has been extended 
from the school district level to the school building level. As one c .servor puts it: 

The classic debate regarding levels of authority and devotion to local control has 
expanded recently. Historically . . . [the] concern was freedom from 
encroachments from federal or state government. Now district level authorities 
are concerned about encroachments from below, i.e., citizen advisory 
committees at the building level and principals who believe in the concept of 
site management (Campbell 1980). 

But it is at the state level that perhaps the most profound changes are taking place. New 
configurations of political power have emerged In state after state, governors and legislators have 
superceded the traditional custodians of educational legislation and are assuming an increasing role 
in the decisions about the financing and control of the schools. As one looks across the Nation, 
one sees that policy decisions about education are more and more being hammered out in legislative 
halls and chambers, in governors' offices, in state board rooms, in the offices of associations and 
interest groups, and less and less in the offices of local school superintendents and the 
meeting rooms of local school boards. 

Over the past fifteen to twenty years, the Nation also has witnessed an unprecedented 
involvement in public education by the federal government- the courts, the Congress, and the 
executive agencies. We now have a cabinet level Department of Education. And despite its 
traditional junior or minor role in school finance, the federal government has become a significant 
force in American education. At the K- 12 level, federal expenditures rose from $642 million in 
1960 to over $15 billion in 1985--a twenty-three fold increase and a significant amount of money, 
even if it covers only 7 to 8 percent of the cost of operating the public schools. And the 
Congress has not been content to play the silent banker, it also has directed how school districts 
should spend the funds. Furthermore, in its oversight role, the Congress has called on the federal 
adminstrative agencies to monitor closely the expenditure of those funds. While recent efforts of 
the Reagan Administration to return more responsibility and decision making authority to 'he states 
have achieved some modest successes, it appears that the federal government's involvement in 
education is not likely to diminish significantly in the immediate future. 

Thus, at this time in our history, we find that the policy process -the power structure of 
American education-is no longer a tightly-knit or closed system. Intervention in the policy process 
has become generally open to any individual or group who can claim to represent a constituency, 
who can gain access to information, and who is familiar with the points of access into the system. 
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Nor is the educational policy process monolithic, fixed, or static. Educational decision making has 
become an evolving, interactive process open to external ideas and influences, involving many 
individuals and groups, involving all levels of government, and all levels of organization and 
program administration. But it is not only the locus of educational decision making that has 
changed. The cast of educational decision makers has been greatly enlarged-parents, other 
citizens, representatives of interest groups, educators, executive staff executives, legislative staff 
legislators, governors, Congressional staff all have become participants in the process. 

C The Catalytic Role of " A Nation at Risk" 

As the Nation approached the middle years of the 1980's, reform was in the air. Study 
commissions were being established. States were considering major reform efforts. But it was the 
Report of The National Commission on Excellence, A Nation at Risk , that truly caught the public's 
eye, moved education to the "front burner" as a critical public policy issue, and served as the 
catalyst for a spate of reform activity across the Nation. Since the publication of A Nation at Risk 
in 1983, the country has been literally besieged with calls for educational reform, calls that have 
issued from national, state, and local levels, from public and private sectors, from academe, from 
business and industry, from parents and other inaividua^s-in short, from virtually all quarters. 
Governors have created their own committees on excellence, state legislatures have proposed and 
enacted a bevy of reform statutes, state boards of education have issued blueprints for action, local 
school districts-often in concert with local business and industrial interests-have established their 
own reform committees. One observer describes the phenomenon as "a kind of rising Greek chorus 
of educational reform that is sweeping across the Nation." 

The reforms, and calls for reform, cover a broad range. In a great many states and locales 
pupil testing has taken center stage for any number of different reasons-to identify curricular 
strengths and weaknesses, to allocate resources, to select pupils for remedial assistance, to evaluate 
school and program effectiveness, to certify promotion and graduation, and to report status and 
progress to the general public. But pupil testing is only one target of current reform efforts. Merit 
pay systems and career ladders for teachers also have caught the reformer's and the public's eye. 
The Commission on Excellence urges that salaries of teachers "be professionally competitive, 
market sensitive, and performance based" (1983). A second national report, the Twentieth 
Century Fund's Making the Grade , proposes "reconsideration of merit-based personnel systems 
for teachers (1983). A third major report, Action for Excellence , issued by the Education 
Commission of the States, advocates development of "ways to measure the effectiveness of 
teachers and reward outstanding performance (1983). 

The high school curriculum also has become a priority target for reform efforts, most visibly 
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in the Commission on Excellence's recommendation that all students seeking a high school diploma 
be required to "lay the foundations in the Five New Basics H -to take 4 years of English, 3 years of 
Math, 3 years of science, 3 years of social studies, and 1/2 year of computer science (1983). This 
particular recommendation, coupled with the Commission's call that "significantly more time be 
devoted to learning the New B^ics (1983) have served as an impetus for many states, as well as 
hundreds of local school districts across the land, to propose and in many cases adopt new high 
school graduation requirements and extend the length of the school day if not the school year. 

Another abiding theme reflected in current nationwide reform efforts is the training and 
retraining of educational leaders-the n ind women who fill the administrative and managerial 
roles of public and private education in America. This theme, in particular, is often woven together 
with a second theme-joint, cooperative efforts between the education and business communities. 
Business and industry are looked to not only as the consumers of the schools' products, i.e., 
educated and trained workers, but also as valuable resources in the design, implementation, and 
support of training programs aimed at developing excellence among education's administrators and 
managers and the school programs for which they are responsible (1983). 

Parental voice and parental choice have become a cause celebre of many of the reform 
efforts. There is a growing feeling that American society has surrendered too much responsibility 
for schooling to governmental bureaucracies and professionalized institutions and, thereby, 
neglected the more human-sized groups, what Peter Berger and Richard Neuhas have called the 
"mediating structures" of society-such as families, communities, voluntary organizations, and 
religious groups (1977). 

There are numerous other targets of current reform efforts-revitalization of teacher training 
institutions, improvement of knowledge dissemination practices, integration of education and 
technology, better fits between education and employment, to name a few. And as we noted 
earlier, even though A Nation at Risk served as a major catalyst for the current reform movement 
that has swept the country, the calls for educational reform and the responses to these 
calls did not necessarily await the publication of A Nation at Risk . A good many reform efforts 
already were well underway before the Commission on Excellence sounded its general alarm. For 
example, the effective schools movement, which has been a powerful driving force for change in 
the schools and, at least in the minds of some, has been eminently successful in improving school 
learning for a great many youngsters, considerably pre-dated the release of the Commission's 
report as well as the several other reform reports which have received nationwide attention. 

But irrespective of the sources of the impetus for any particular reform effort, what appears 
central to all of them is an almost universal call for data-data on how students are doing in our 
schools, on what they are learning, on their levels of achievement; data on how teachers are doing, 
on what constitutes good teaching, on the mix of conditions necessary to ensure that our 
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professional leaching ranks become filled with "the best and the brightest;" data on curricular 
programs, on effective instructional practices, on new ways of learning; data on the context in 
which schools operate, on the climate in the classroom, on family, social and economic 
environments; data on resources, on the most effective mixes and uses of resources, 
on resources and equity issues; data on alternative approaches to schooling, on the private sector of 
schooling, on choice within the public sector. 

We see, then, as we enter the final years of the decade of the 1980's, American education- 
particularly as it takes place in c s elementary and secondary schools-deeply immersed in a reform 
effort similar to wha* the Nation experienced following 'Jie general alarm raided in 1958 by the 
Russian launch of Sputnik. Whether one fully agrees with all of the cries of alarm and calls for 
reform now being sounded across the country, one has to acknowledge that American 
education— particularly at the elementary and secondary level-currently is in a state of ferment and, 
consequently, open to massive changes in its traditional modes of operation. As noted in 
the Synthesis Report , Usdan, in citing Kirst's attention cycL, contends that we have passed 
through the stage of "alarmed discovery" and are now in the mid t of a second stage of "crisis 
activity" (Gwaltney and Balcomb 1981). However, as Usdan further contends, the reform 
movement can only endure if the effectiveness of specific reforms can be prove J (1981). But to 
demonstrate the effectiveness of the reforms, one needs information substantially diffeient from 
that which we traditionally have gathered, and that information needs to be provided to a 
substantially different group of users. 



D. New Users of Educational Information 
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As we noted earlier, in the past the primary information users were the education 
professionals responsible for conducting the educational process. These decision makers 
participated in local information systems focused around day-to-day decisions made by 
teachers and administrators. Such information systems generally were limited to providing basic 
data on the local level about pupils, personnel, educational services, and finances. Now, however, 
we are finding that information must be conveyed beyond the boundaries of the local educational 
agency. Parents need information about the character and quality of the educational alternatives 
available to their children. The citizenry-local, state, and national-requires and is demanding 
information about the quality of its schools. The busines: c amunity~at local, state, and national 
levels -is calling for systems that will provide quality assurance information on the schools. State 
officials-governors, legislators, state schc 1 board members officers of state education 
agencies-require more comprehensive and accurate information on the quality of the schooling 
being conducted within their purview. Federal officials- from the President through the Congress 
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to the agency administrators-are seeking information on the general condition of American 
education, as well as on the effectiveness of federal education policies and programs 

We alluded earlier to a growing idling among the body politic that American society has 
surrendered too much responsibility for schooling to governmental bureaucracies and 
professionalized institutions. There is a rising demand for parental voice and parental choice in the 
education of the young; in many locale*, parents have moved to excercise more voice and more 
choice in these matters. As a result of becoming increasingly important actors in the policy 
process, parents also have become information users. They have sought information to help them 
fashion plans for decentralization, school site management, parent advisory councils, 
and other proposals designed to get around the entrenched school bureaucracy. They have sought 
information to help them make choices about the schools to which they send their children, choices 
within the public sector as well as choices betweeen the public and private sectors. But too often 
the needed information has been lacking. James Coleman, in his invited paper written for the 
redesign project, argues that NCES data activities can be used to help correct this situation, to 
augment parental resources by "encouragment and facilitation of parental and community use of 
information about student performance and school functioning." In Coleman's view, NCES ought 
to act, in effect, as "a representative of the consumers of education," informing parents of their 
information rights with respect to both public and private schools, providing to local districts which 
are so inclined specifications for appropriate consumer information systems, and designing 
consumer information systems to accompany school choice plans developed for use within public 
school systems, as well as those that include nonpublic schools (1985). 

The general citizenry-as well as parents—also have become growing users of information on 
the schools. They want to know what the school 0 are doing. They want to know what and how 
well children and young people are learning. Citizens want to know if the schools are equipping 
the young for productive employment. They want to know it the schools are developing in the 
young an understanding and appreciation of our democratic form of government. They want to 
be able to compare their local schools with other schools, their state educational system with other 
state systems, the Nation's schools with the schools of other nations. They want to be assured that 
their tax dollars are being well spent. In short, they want to know how well the educational system 
is working, how far it is from attaining excellence, how far it is from attaining equity. As James 
M. Banner, Jr., in his invited paper put it: 

For good or ilJ, the American public now seeks to be assured of improvements in 
education at all leveiS, especially in the primary and secondary schools. And, 
characteristically, it wants information that compares present conditions with those 
of the recent past and conditions in one jurisdiction with those in others. Yet the 
plain fact of the matter is that the data available to provide such comparisons v: 
embarrassingly weak. The public is being mislead by their use (1985). 
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It is abundantly clear that citizens are calling for more comprehensive and moie accurate 
information on their schools-information to use in their educational policy making roles as persons 
interested in the general condition of the schools, as voters on local tax issues, and as members of 
interest groups working to influence policy development at local and state levels if not the 
national level. 

The business community, too, has become a user of educational information. It wants 
assurance. It wants assurance that fair and objective data on the perfomance of teachers are 
available. It wants assurance that information on the certification of teachers and administrators is 
available. It wants assurance that rigorous criteria for selecting and retaining teachers are being 
adopted and implemented It wants assurance that fair and objective data on student performance 
are available. It wants assurance that the schools are preparing the young "for productive 
participation in a society that depends ever more heavily on technology, . . . "(1983). And the 
latest entry in the long list of reports on American Education, by the blue-ribbon Committee for 
Economic Development (CED), corroborates this contention (1985). Dennis Doyle and Marsha 
Levine, in a recent article describing the CED report, write that: 

There is an opportunity today to forge a new quid pro quo b etween Americans and 
their public schools. There will be more money for education when there is more 
education for the money. The business community (at least, as represented by the 
CED trustees) stands ready to put its shoulder to the wheel to support public 
schools— including substantial increases in funding-when the public schools are 
willing to set and meet the objective of a high-quality education for ever} citizen 
(1985). 

It goes without saying that, if the business community is going to make good on its comnrrment, it 
will need adequate, accurate and timely information on the progress of the schools. 

State officials, as education has become more centralized and governance and control 
mechanisms traditionally left to local school boards have reverted to state school boards and state 
education agencies, if not to state legislatures and governors' offices, also have become increasing 
users of educational information. To a great extent the increasing call for and use of education data 
ha\ been driven by the states' assumption of a more active role in the decision making process; but 
equal weight needs to be given to rising concerns about educational accountability. Legislatuies are 
increasingly questioning the effectiveness of America's public schools; and increasingly they 
have assumed a more active role in education decsion making. Fuhrman and Rosenthal argue that 
. . legislatures have taken on the role of pre-eminent education policy-makers in some states; in 
many others they are at least co-equal partners; and in only a few are they still secondary (1981). 
As such, legislators have made increasing demands on state agencies for more accurate, more 
comprehensive, and more timely data on the quality of public schooling. Governors, too, have 
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become ardent consumers of educational information as they have moved to enter the education 
policy arena and mount their own reform programs. They, like the members of the state 
legislatures, are no longer content with the traditional education information repons preferred in 
past years. They want information on the effectiveness, on the costs and the benefits, of the 
policies and programs mounted to improve the quality of schooling. 
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Chapter 2 



Capabilities of Present Education Data Systems 

In the previous chapter we have documented widespread demands for the reform of 
American education, and the myriad ways state governments, local education agencies, citizens' 
groups, and business and professional organizations are working to meet those demands. Coupled 
with demands for reform and actions to bring it about are new and increasing requirements for 
information on the status, condition, and functioning of schooling in the United States, and for 
information on education more broadly conceived In this chapter we examine the capabilities of 
current educational data systems to meet these new requirements, and the consequent needs for 
reform of federal activities in the collection and dissemination of educational data. 

At the core of new requirements is the need for an integrated program of data collection, 
analysis, and reporting, in contrast to the largely unarticulated set of data collection projects that 
presently operate at all levels of the education enterprise. Although major factors that influence the 
operation of the nation 's education systems are inextricably linked ; e.g., changes in levels of 
educational resources produce changes in the availability and quality of education personnel, and 
these, in turn, produce changes in educational offerings and services, present data-collection 
projects are inadequate to describe and characterize such linkages. In many cases, the structure and 
articulation of interrelationships among components of the nation's education systems must be 
deduced from unexamined inferences or from incomplete and inappropriate empirical findings. No 
education data system or program adequately characterizes the whole of the education enteiprise. 
Although relationships between educational resources, expenditures for personnel, and the quality 
of personnel are known in part, there is no integrated educational data system to document the 
extent and operation of such relationships, either for the nation as a whole or for individual states. 

A. The Structure and Conte nt of the Nation's Education Data Systems. 

Responsibility for the collection and dissemination of information about education in the 
United States is fragmented. At base, much of the information currently collected is derived from 
administrative records. Most of these records are maintained by local education agencies and 
schools. 

Local Education Agencies Formal records in local education agencies are usually of six 

types: 

* pupil records (e.g., cumulative folders, transcripts, etc.); 
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* instructional service records (e.g., curriculum specifications, courses offered, time 
schedules, lists of authorized textbooks, syllabi, etc.); 

* personnel records (e.g., occupational categories, specific assignments, certification levels, 
college transcripts, application forms, etc.); 

* financial records (e.g., accounting journals and ledgers, payroll records, etc.); 

* records required by other agencies (e.g., health records, state- and federally-mandated 
records, etc.); 

* policy records (usually special data collections or tabulations of administrative record data 
for policy formulation or administrative decision-making) 

These records are the formal outcomes of "official" events that occur within the local education 
agency or within the schools operated by the agency. When students enroll, teachers are hired, 
purchases are made, grades are issued, or tests are taken, records are created, supplemented, or 
updated. Some of these records are initially maintained in separate school files e.g., student 
grades - and then formally summarized and entered into central record systems. Unfortunately, the 
detailed content and organization of record systems vary from one local education agency to 
another. Both historically and currently this has caused massive reporting and summarization 
problems as records are transferred from local education agencies to state agencies or to other 
jurisdictions; inconsistencies in the types and forms of records kept by different local education 
agencies produce substantial differences in the meaning of data reported by those agencies. 
Common units of reporting and common definitions are necessary precursors of useful data 
aggregations, and these common elements often do not exist or are not used. The structure of 
record systems maintained by U. S. high schools, and the problems inherent in such systems, are 
discussed at length by Coleman and Kanveit (1972). 

The records maintained by local education agencies are seldom linked in any formal way. 
For example, although students' secondary school transcripts indicate the courses these students 
completed in a specific semester, they typically do not indicate which teachers taught the courses. 
And even when teachers are linked to courses, there is usually no feasible way to link this 
information to the teachers 1 certification levels or college transcripts. 

State Education Agencies, Local administrative records form the primary basis for reporting 
information to state education agencies. States often control the form and content of these 
records (e.g., The Illinois School Student Records Act - P. A. 79-1108). Usually a state education 
code specifies the content and timing of required reports to the state education agency. In 
California, for example, the State Department of Education issues a Data Acquisition Calendar 
twice a year covering state-mandated data collections and reports by local education agencies. This 
calendar also informs local education agencies that no data collections other than those listed are 
required by the state. In 1984-85, these semi-annual calendars contained twenty and nineteen 
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pages, respectively. 

One primary purpose of local agency reports to states is to determine the amounts of state tax 
monies local agencies receive in the form of general aid and program-specific funding. A second 
purpose is to monitor the compliance of local agencies and schools with the provisions of then- 
state's education code. The information contained in these reports forms the core of the states' 
information systems about local education agencies and school:. However, this core is often 
supplemented by special-purpose periodic or ad hoc data collections conducted by the state 
education agency. A prominent example of such data collections is state testing or assessment 
programs. In most states, the results of special data collections, including student testing programs, 
are not integrated into a comprehensive educational information system. Typically, different 
subdivisions of the state education agency are responsible for, and use, different categories of data. 
Most often, distinct data files are maintained for the various purposes of the subdivisions. 
However, as new information technology has become available, some states have begun to design 
and operate more highly integrated educational information systems. 

The U. S. Department of Education. The data-collection projects of the National Center for 
Education Statistics are central to the federal role in the collection, analysis, and dissemination of 
information about elementary and secondary education. 

The current data-collection activities of the National Center for Education Statistics (as of fall, 
1985) can best be described as a discrete set of projects, in contrast to a data program or system for 
providing information on the status of education in the United States. Funk and Wagnalls N;w 
Practical Standard Dictionary defines a program as "Any prearranged plan or course of 
proceedings," and the Oxford English Dictionary defines a system as "An organized or connected 
group of objects; a set or assemblage of things connected, associated, or interdependent so as to 
form a complex unity : a whole composed of parts in orderly arrangement according to some 
scheme or plan" (emphasis added). That the current data collection activities of NCES form 
neither a program nor a system is amply documented in the public discussion draft of the Synthesis 
Report produced for the NCES Redesign Project, and in the 60-odd papers that underlie that 
draft. Hence our use of the term "projects" in describing current NCES activities. 

NCES collects data on education from two primary sources. Contracts with most state 
education agencies and less-formal arrangements with others provide for acquisition of a "Common 
Core of Data" that is secured and reported by states from the administrative records of local 
education agencies and the State's own records. These data are provided annually by every state 
education agency for a census of local education agencies in the United States. The quality and 
completeness of these data depend on the skill and interest of personnel in local and state 
education agencies, and on the provisions of state regulations concerning data required from local 
education agencies. NCES serves as a recipient and compiler of the Common Core of Data, 
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rather than as a proactive collector of the data. It should also be noted that NCES does not conduct 
systematic audits of the data supplied by the states. 

The second principal source of NCES data is a set of occasional and periodic cross-sectional 
and longitudinal sample surveys of local education agencies, school administrators, teachers, 
students, and parents. NCES survey data are, almost exclusively, collected by mail using 
pencil-and-paper questionnaires. Some NCES surveys are periodic (such as biennial surveys of 
public and private schools, conducted in alternate years), and others are occasional (such as a 
survey of recent college graduates who have received bachelors and masters degrees). In contrast 
to the Common Core of Data secured from state education agencies, data NCES collects through 
sample surveys are, with rare exception, obtained from nationally or possibly regionally 
representative samples of individuals, institutions, or agencies. These samples generally support 
estimaron of parameters for a nationwide population of units, rather than estimation of 
state-by- state parameters. For some surveys, such as the National Assessment of Educational 
Progress (NAEP), reasonably precise estimates of parameters can be computed for regional 
populations. 

A concise listing of the types of data collected through the current projects of the National 
Center for Education Statistics is provided in Appendix C of the Synthesis Report . The listing 
includes the population of inquiry, coverage, source, summary level, periodicity, and variables for 
which data are collected in every component of the Common Core of Data, in each of the major 
sample surveys NCES has conducted since 1980, and through the October Current Population 
Survey operated by the U. S. Bureau of the Census 

B. The Ability of Current NCES Data Projects to Meet New D emands for 
Information 

Data Quality . According to authors of papers for the NCES Redesign Project, the 
deficiencies of data collection projects operated by NCHS as of the spring of 1985 are legion. 
NCES data on the nation's systems of elementary and secondary education are claimed by these 
authors to be 

* inaccurate (David, 1985, pp. 2-3; Eubanks, 1985, p. 1; Hawley, 1985, p. 2; McClure and 
Plank, 1985, p. 1; Murnane, 1985, p. 6; Walberg, 1985, p. 20ff; McDonough, 1985, in a 
letter); 

* imprecise (Barro, 1985, p. 16; Harrison, 1985, p. 2; Hawley, 1985, p. 19; Hilliard, 1985, 
pp. 1 1 and 15; Lehnen, 1985, p. 4; Murnane, 1985, p. 2; Rosenholtz, 1985, p. 2; 
Selden, 1985, pp. 15 and 17; Thomas, 1985, p. 5; Valdivieso,1985, pp. 1 and 13) 

- inadequate in scope and coverage (Banner, 1985, p. 7; Barro, 1985, pp. 2,4 and 9; 
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Bishop, 1985, p. 18; Eubanks, 1985, p. 2; Hannaway, 1985, p. 1; Hersh, 1985, p. 11; 
Miller, 1985, p. 1; Natriello, 1985, p. 3; Odden, 1985, p. 5; Peterson, 1985, pp. 11 and 12; 
Reisner, 1985, p. 2; Scott-Jones, 1985, p. 5; Selden, 1985, p. 1; Thomas, 1985, 
pp. 2ff.); 

* ina ppropriate (Banner, 1985, p. 6; McClure and Plank, 1985, pp. 10 and 14); and 

* lacking in timeliness (Berrvman. 1985, p. 17; Grant, 1985, pp. 2 anJ 3; Harrison, 1985, 
p. 2; McClure and Plank, 1985, p. 14; Reese, 1985, letter). 

On the last point, W. Vance Grant, head of the NCES Statistical Information Office, compiled a 
list of the most recently available statistics that can be used to fulfill requests for information most 
frequently sought from his office. As of July 1985, no data more recent than Fall 1983 statistics on 
enrollments by grade were available tc satisfy such requests. The most recently available 
information on school employees was, for most data items, for the 1979-80 and 1980-81 school 
years, with limited data available for Fall 1981. A great deal of data on pupils, such as school 
attendance and membership, and enrollments in high school subjects, were most recently available 
for the 1980-81 or the 1981-82 school years. Fiscal data, including information on local education 
agency revenue receipts and expenditures, were available most recently for the 1982-83 school 
year, but many data items, such as expenditures for salaries of nonprofessional school staff and 
expenditures for school library books, instructional supplies, and other instructional expenses, had 
not been compiled for school years later than 1975-76. 

Judging the accuracy of statistics reported by NCES is difficult, since clearly-correct 
standards are often unavailable. Nonetheless, the examples of questionable results cited by Cooke, 
Ginsburg and Smith (1985), and Plisko, Ginsburg and Chaikind (1985) bring to mind the old 
adage "When the clock strikes 13, you begin to doubt the clock...". These papers raise questions 
about the validity of data supplied to NCES by other agencies, as well as the validity of data 
collected by NCES. In some instances, problems of definition lead to markedly different reports 
on purportedly identical variables. Cooke, et al. (1985, p. 8) cite the nationwide high school 
dropout rate, reported as 27 percent by NCES, and only 16 percent by the Census Bureau. Plisko, 
et al. (1985, p. 10) rightly question the large variability among states in the percentage of enrolled 
students who are classified as "special education students." States reported as few as 5 percent and 
as many as 1? percent of their enrolled students in special education categories. These authors 
(Plisko, et al., 1985, p. 10) conclude: "There is no physiological explanation that could account for 
these report [sic] differences exceeding 100% in the prevalence of handicapping conditions." 

Plisko, et al. also report highly implausible year-to-year and state-to-state variations in basic 
statistics that compose the Common Core of Data. Although the National Education Association 
(1985) refers to the Common Core of Data as the "cornerstone of educational information in the 
United States," Plisko, et al. note such anomolies in Common Core of Data statistics as variations 
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in pupil-to-staff ratios of 140 percent across states within the same academic year, and wide 
variations in pupil expenditures within states from one year to the next 

An obvious data quality problem arises from the dependence of the U. S. Department ot 
Education on the accuracy of statistics reported by state education agencies. In such 
programs as the Common Core of Data and the Vocational Education Data System (VEDS), the 
federal agency merely aggregates, compiles, and reports the data supplied by state agencies, in the 
absence of an effective auditing or quality control function. Cooke, et al. (1985, p. 6) cite as 
flagrant inaccuracies, the State of New Jersey reporting 50 percent more students enrolled in high 
school vocational education courses than that state's total population of high school students, and 
the State of Virginia reporting that three times as many American Indian students were enrolled in 
vocational education courses, than were in the entire Indian population of Virginia. Whether these 
reports are necessarily in error is difficult to determine, since the definition of "course enrollment" 
is not made clear. It is impossible to determine whether these states were providing duplicated 
counts of individual students who enrolled in a number of different vocational education courses 
during the same academic year, or were providing unduplicated counts of vocational education 
students, regardless of the number of courses in which they enrolled. If the former definition were 
used, the results cited by Cooke, et al. are still implausible but not impossible. If the latter 
definition were used, the results are clearly impossible. 

Scope and Coverage of Data . Many authors of papers for the NCES Redesign Project cited 
inadequacies in the scope and coverage of data presently collected, and the inability of the present 
NCES projects to provide information that can be used to compare the condition and progress of 
education in the various states. The latter point will be considered first. 

With the exception of the Common Core of Data, most of the data-collection activities of 
NCES support estimates of parameters only for nationwide, or occasionally for regional, 
populations. Yet education is an activity that is constitutionally reserved to the states and, as noted 
by many of the contributors to the NCES Redesign Project, the vast majority of important 
education policies originate at the state level of government The need for education statistics that 
support comparisons across states has been strongly voiced: 

Lehnen (1985) stated: 

"National averages and other statistics do not reveal much about the state education systems 
... Yet it is the states who will determine the direction and scope of education policy and not 
the federal government Without this detail NCES data will have only limited utility for 
policy studies within the states." 

And the National Governors 1 Association (1985) noted: 

"In order to perform education policy setting functions, states need to plan, develop, 
implement, and evaluate education initiatives. ... national trend data and consistent and 
accurate data from all states for macro comparison purposes is ~f key interest ... samples 
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should be examined to determine the feasibility of expansion to collect data more state 
specific." 
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Finally, Odden U985) suggested: 

"There is no question that the state is the primary actor in education policy federal data 
collection should reflect this fact Thus data should be collected on a district and state basis; if 
a sample of district data are collected ... the sample should be REPRESENTATIVE FOR 
EACH OF THE FIFTY STATES." 

Deficiencies in the scope and coverage of statistics presently reported by NCES are well 
summarized in the Synthesis Report , and span the content of all reasonable models for 
structuring data on status and change in U. S. education. Of even greater concern is the paucity of 
information essential to an assessment of the need for, and the consequences of, the reform actions 
that are currently reshaping American education. 

In the first chapter of this report, we described the rapidly-changing face of U. S. education 
policy. A plethora of reports on the quality of American education, including those of The National 
Commission on Excellence in Education (1983), The Education Commission of the States (1983), 
The Twentieth Century Fund (1983), and The Carnegie Foundation for the Advancement of 
Teaching (Boyer, 1983) have called for major reform of public school resource allocations, 
curricula, requirements, instructional patterns, certification routes, etc. Many authors of papers 
prepared for the NCES Redesign Project document the inability of the current NCES data-collection 
projects either to substantiate the need for recommended reforms, or to support an investigation of 
their effects, should they be put into practice. Deficiencies cited wer ^ pertinent to analyses 
of: 

* the allocation of educational resources 

Available data on the allocation of educational resources among school systems within states, 
and among categories of students are meagre and outdated Far more information is needed on the 
magnitude and distribution of various categories of resources to school systems, to schools, and to 
categories of students. 

For example, Barro (1985,pp. 4-5) notes that: 

"The NCES currently produces what might fairly be described as skeletal information on 
school finance. ... There are no NCES publications describing the distributions of revenues 
or expenditures among local school districts, either nationally or within states, even though 
such distributions (e.g., intrastate disparities) have long been the central concern of school 
finance policymakers and researchers." 

* investments in and expenditures for education 

Educational expenditure data collected by NCES are currently available only for gross 
categories of expenditures -"instruction, support services, and non-instructional services" -- that 
mask, rather than inform an accurate picture of the utilization of resources in school systems and 
schools. Again, Barro (1985; pp. 4-5) provides a useful synopsis of the deficiencies of 
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currently-available data: 

"The education expenditure data currendy reported by NCES are serviceable at best, for 
making gross fiscal comparisons among states and examining broad trends in public support 
for the schools. Even in those applications they can be misleading, because differences in 
dollar oudays among states and over time do not necessarily correspond to differences in 
educational resources. ... Two reasons for the limited usefulness of current data are that 
expenditure data are not collected in sufficient detail to be connected with resource categories, 
and expenditure and resource categories are not coordinated. Consequendy, information on 
dollar oudays cannot be linked to anything real. Most expenditures of direct educational 
interest, in fact, are contained within the single, overbroad, traditional category, 'instruction 
which is not decomposed either by type of resource or by the various purposes for which 
instructional resources are used." 
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Odden (1985; pp. 4-5) provides recommendations that are consistent with those of Barro. In 
response to new education reforms, he cites the need for detailed, state-comparable expenditure 
data by function (e.g., administration at the school system central office level, administration at the 
school building level, administration of special programs, instruction, transportation, and operation 
c nd maintenance of plant), and by program (e.g., regular education, programs for special-need 
students, categories of curriculum such as mathematics, reading, science, social studies, etc., and 
level of education, such as elementary, middle, and secondary). Odden notes that such data are 
central to emergent policy interests at both state and federal levels. 

* t he coverage and scope of curricula 

Several authors noted that NCES currendy collects little information on what is taught in 
schools, despite the her"v emphasis of the "reform reports" on strengthening school curricula. 
Hawley (1985) cited the absence of data on curricular scope and the level of difficulty of subjects 
offered in elementary and secondary schools. Cronin (1985) expressed concern that the 
suggestions for curricular reform contained in A Nation at Risk would lead to an overly narrow 
curriculum, but cited the paucity of NCES data that could be used to document current curricular 
breadth or curricular reform, should the nation's schools decide to follow the suggestions of the 
National Commission on Excellence. 

* the nature of educational requirements and standards 

Although the 1983 report of the National Commission on Excellence in Education 
recommended substantial increases in high school graduation requirements, NCES does not 
regularly collect data on current graduation requirements in school systems and states, nor on 
standards of quality imposed on school curricula or student performance. The High School and 
Beyond study 

collected data on the course-taking patterns of high school graduates, but these data were collected 
prior to the National Commission report and no comparable data that would support 
analyses of school systems' and students' responses to the National Commission report are 
currendy collected by NCES or its contractors. Data on Carnegie unit requirements, reported by 

34 

38 



state in the 1985 edition of The Condition of Education were collected and reported by the 
Education Commission of the States. 

Minimum competency achievement testing is another recent reform that is poorly understood, 
and for which impact data are neither collected nor made available by NCES. Although The 
Condition of Education (pp. 68-69) reports that competency testing was used as a high school 
graduation sanction in 24 states as of 1984 (based on data supplied by the Education Commission 
of the States), available data will not support analyses of the relationship between the use of 
competency tests and high school dropout rates, changes in high school curricula, or changes in 
students' patterns of course-taking. 

* the extensiveness and quality of instruction 

Not only are data on the structure and extensiveness of curricula in the nation's schools 

generally unavailable, we have even less information on what students are doing in our schools. 

As Peterson (1985; pp.3-5) noted: 

"To take the pulse of American education, we need to know what students are doing and 
learning in classrooms in the United States. ... Following publication of the Nation at Risk 
report in 1983, many states responded to the recommendations by lengthening the school 
day, many school districts set minimal standards for the number of minutes that teachers 
must spend teaching each of the major subject areas during a given week. The impact of 
these new guidelines on what teachers and students are doing in classrooms has not been 
assessed. 11 

Peterson also cites the lack of data on the quality of educational activities in our nation's schools. In 
particular, she notes the inability of a national commission on reading to detemine the amount of 
time teachers typically spend on the most important structural components of reading, such as 
phonics in the early grades or to determine the amount of time students spend engaged in such 
essential learning activities as silent and oral reading. Data on the amount of time students spend 
doing "seatwork" were collected in some local studies in the late 1970's, but no information is 
available on whether these students were engaged in any useful educational activity or were merely 
wasting time. 

Hersch (1985) also described the importance of time devoted to instruction as a component 
of school effectiveness, and then noted the absence of NCES data on the distribution of 
instructional time across subjects and topics. 

* the demographics of pupil populations 

Several contributors to the NCES Redesign Project noted deficiencies in data presently 
available on the demographics of pupil pooulations. Although he credits NCES with some of the 
most extensive data available on Hispanics, Vakil vieso (1985; pp. 11-13) cites the need for more 
extensive, and better-differentiated data on the educational characteristics and participation of 
Hispanics. He notes that NCES uses a general category labeled "Other Hispanics" in its 
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data-collection and reporting and, in so doing, combines data for populations with very different 
cultural and educational backgrounds and needs. Valdivieso also calls for collection of data by 
"racial/Hispanic indicators" in the Common Core of Data. 

Hilliard (1985, pp. 4-6) suggests that demographic data on pupil populations are essential tc 
investigations concerning problems of equity in our nations schools. He notes that 
presently-collected data do not support analyses of inequities in the allocation of school resources 
to students of different racial and ethnic groups, nor do they allow satisfactory investigations of the 
educational experiences of students of different racial and ethnic groups. 

Eubanks (1985, p. 3) suggests the need for better data on the relationships between students' 
social classes and their participation and success in the nations schools. This need for additional 
data is also stated by Thomas (1985, p. 3) when she notes that schooling in the United States is still 
highly stratified by students' race and social class, and that data on such stratification "do not exist 
on a national basis." 

* the co n text of education, both in and out of schools 

Coleman (1985) builds a strong argument for the changing nature of educational demands 
placed on schools as a result of changes in family and community structure in the United States, 
including the prevalence of working mothers, and one-parent families, and the growth of 
"intergenerationally separate" social structures. To assess the need for school policies that meet 
these new challenges and the schools' success in doing so, Coleman calls for the collection of a 
wealth of contextual data not presently available from NCES, including data on the characteristics 
of students' families, data on the schools' relations to students' families, data on community 
organization, and data on the schools' relations to their communities. 

Usdan (1985) suggests that data on education must go well beyond data on schools, and 
must include information on the societal context in which schools operate. He notes the increasing 
delivery of educational services by agencies and institutions outside the schools, such as private 
sector organizations, the military, and voluntary associations, and suggests that information on the 
extensiveness and quality of education services provided must be collected from these agencies 
and institutions, as well as from the schools. Usdan further notes the need for "social data" on such 
issues as the increased burden placed on the schools by single parents and the prevalence of 
"latchkey children," and the relationships between education and economic trends and employment 
opportunities in the communities served by the sC ools. 

* the characteristics and quality of educatio 1 personnel 

Perhaps the most dramatic changes in educauon policy can be seen in states' reactions to the 
public clamor for higher quality teachers. In recent years a solid majority of the states have 
adopted new and stiffer requirements for teacher certification, including demonstration of content 
knowledge and pedagogical knowledge by passing one or more tests, and demonstration of 
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pedagogical skills by earning passing marks in an on-the-job evaluation. Incentives for joining the 
teaching profession have also been provided, through such major reforms as revision of salary 
schedules, ac option of merit pay plans, and establishment of career-ladder programs. 
According to Anrig (1985): 

"One of the fastest-moving changes in this period of educational reform is in teacher testing. 
In as little as five years, state-required testing for aspiring teachers to enter preparation and/or 
to become certified has spread from a handful of states - mainly in the southeast — to a 
nationwide trend involving 38 states, with seven additional states currently considering a 
teacher testing requirement. In 1984 alone, nine states enacted teacher testing laws or 
regualtions." 

Similar statistics are documented by Goertz and Pitcher (1985) and by Sandefur (1984). 

Again, current NCES data-collection projects do not provide the kinds of information that 
must be available to assess the neer 1 for, and the impact of, these massive educational reforms. 
The Common Core of Data provides simple counts of full-time-equivalent staff employed in public 
elementary and secondary schools, and in public school systems, by state and by type of 
assignment (e.g., principal, assistant principal, classroom teacher, etc.). Although a recent sample 
survey will eventually produce statistics on teaching assignments, teacher training and teacher 
experience, at present data collected by the National Education Association provide the only 
information on such basic characteristics of the jistructional workforce. The most recently 
published Digest of Education Statistic? contains basic demographic and educational data on public 
school teachers (such as median age, race, gender, highest degree held) and a percentage 
distribution of public school teachers across subject areas, but all such data are for the 1980-8 1 
school year, and all were provided by the National Education Association. To understand the need 
for, and the effects of, reforms in teacher selection, employment, and compensation, NCES must 
collect and report information on the recruitment and retention of teachers, in addition to statistics 
that will inform judgments on the quality of the instructional workforce. No such data exist for 
the nation, much less for individual states. Since reform actions differ substantially across states, 
state-by-state information will be essential to understanding their impact. 

* the outcomes of education 

Since the late 70's, more than two-thirds of the states have adopted some form of minimum 
competency achievement testing of students. In almost half the states, competency tests are u^ed 
as a high school graduation sanction (Pipho, 1984). Although numerous educational researchers 
have speculated on the effects of minimum competency testing on high school curricula, teacher 
morale, student dropout rates, and school curricula (see Jaeger and Tittle, 1980), data currently 
collected by NCES offer little or no information that can inform such judgments. As we have 
already noted, NTS data provide no information on the breadth and depth of high school 
curricula or on the morale (or consequent career intentions) of teachers. And data on student 
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dropout rate are highly suspect. To examine the effects of minimum competency achievement 
testing, we must have consistent state-by-state data on these factors, collected and reported over a 
number of years. 

In summary, the context of American education is changing rapidly, with new educational 
policy that affects all participants and stakeholders. To understand the need for policy change, its 
short-term impact, and its long-term outcomes, will require a radically improved, and vastly 
modified, national elementary and secondary education data system. 
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Chapter 3 



What Should be the Federal Role in Building a National Educational 

Information System? 



A. The Mission of the National Center for Education Statistics 



A unified and coherent program for acquiring, analyzing and disseminating information on 
the condition of education throughout the United States cannot exist without a centralized 
administrative unit within the federal government that has, as its sole mission, accomplishment of 
those goals. On March 1, 1867, the Congress acknowledged the need for an agency in the 
Executive Branch that would meet the nations needs for information on education (Kursch, 1965; 
pp. 11-12). In establishing a department of education (without Cabinet standing) the Congress 
declared: 

"Sec. 1. There shall be established, at the city of Washington, a department of education, for 
the purpose of collecting such statistics and facts as shall show the condition and progress of 
education in the several states and territories, and of diffusing such information respecting 
the organization and management of schools and school systems, and methods of teaching, 
as shall aid the people of the United States in the establishment and maintenance of efficient 
school systems, and otherwise promote the cause of education throughout the country. 

"Sec. 2. There shall be appointed by the President, by and with the advice and consent of the 
the Senate, a Commissioner of Education, who e hail be intrusted with the management of the 
department herein established, and who shall receive a salary of $4,000 per annum, and who 
shall have authority to appoint one chief clerk of his department, who shall receive a salary of 
$2,000 per annum, one clerk who shall receive a salary of $1,800 per annum, and one 
clerk who shall receive a salary of $1,600 per annum, which said clerks shall be subject to 
the appointing and removing power of the Commissioner of Education. 

"Sec. 3. It shall be the duty of the Commissioner of Education to present annually to the 
Congress a report embodying the results of his investigations and labors, together with a 
statement of such facts and recommendations as will, in his judgment, subserve the 
purpose for which this department is established. In the first report made by the 
Commissioner of Education, under this Act, there shall be presented a statement of the 
several grants of land made by Congress to promote education, and the manner in which 
these several grants have been managed, the amount of funds arising there from, and the 
annual proceeds of the same, as far as the same can be determined. 

"Sec. 4. The Commissioner of Public Buildings is hereby authorized and directed to furnish 
proper offices for the use of the department herein established." 

Thus, from its very beginning, the central purpose anu ocus of the Department of Education 
was the collection, analysis, and reporting of information on the condition and progress of 
v ducation, for the dual purposes of helping states and local school systems improve their 
effectiveness and informing the Congress on the general atus of and returns to the federal 
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investment in education. 

In the General Education Provisions Act of 1974, as amended (Section 406(b), 20 U.S.C. 



"The purpose of the Center shall be to collect and disseminate star, ties and other data related 
to education in the United States and in other nations. The Center shall: 

(1) collect, collate, and, from time to time, report full and complete statistics on the condition 
of education in the United States; 

(2) conduct and publish reports on specialized analyses of the meaning and significance of 
such statistics; 

(3) assist State and local education agencies, including State agencies responsible for 
postsecondary education, in improving and automating their statistical and data 
collection activities; 

(4) review and report on educational activities in foreign countries; and 

(5) conduct a continuing survey of institutions of higher education and local education 
agencies to determine the demand for, and the availability of, qualified teachers and 
administrative personnel, especially in critical areas within education which are developing or 
are likely to develop, and assess the extent to which programs administered in the Education 
Division are helping to meet the needs identified as a result of such continuing survey." 

B. Assumptions Concerning Federal Participation in a Na tional Educational 
Information System 

A central assumption of this proposal for a n^aonal educational information system is 
embodied in its title - that it be a national ii, formation system, not solely a federal information 
system. This objective can be realized only by recognizing that education is a function principally 
reserved to the states and, for many of its most fundamental operations, delegated by the states to 
local education agencies. If information concerning education is to serve as a stimulus for 
reform and a guide to achieving its success, it will be through the efforts of oolicy-makers and 
educators at all levels of government. A national educational information system must be designed 
to serve all of these constituencies. 

Although the federal government has had profound effects on U. S. education in the last 
three decades through a plethora of policies concerning such diverse issues as equality of 
educational opportunity, the burden imposed on local school systems by federal installations, the 
quality of education provided to children of the poor, the educational opportunities and services 
afforded handicapped children, the rights of ncn-English-speaking children to instruction in then- 
native language, and the quality of education provided to children of migrant workers, the federal 
government manages education only on American Indian reservations. In general, the 
management of education is wholly reserved to the states and, through state statutes, delegated in 
part to local education agencies. Therefore, state education agencies and local education agencies 
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have need for management information as well as information on the effects of their education 
policies, while the federal government, apart from its need for small amounts of data to enforce 
regulations concerning the operation of federal education programs, needs policy information. 

The distinction between information required for management and that required for policy 
analysis is more than superficial. To manage effectively, an agency must have comparable 
information on every administrative unit within its purview. To assess the need for, and the effects 
of, educational policies, information on representative samples of units will suffice. The 
substantive distinction between management (a tactical concern) and policy analysis (a strategic 
concern) also implies the need for somewhat distinct types of information. Thus, the content of an 
educational management information system would be somewhat distinct, but far from entirely 
distiuct, from the content of an educational policy information system. 

In an effectively designed national educational information system, the needs of data users at 
all levels of government, in addition to the data needs of consumers of education, would be 
considered. The education information system proposed here is principally designed to meet the 
policy information needs of the federal government, while stimulating, supporting, and 
complementing systems designed to meet the policy information and management information 
needs of state governments and local education agencies. It explicitly seeks to meet some of the 
information needs of consumers of education. 

Past experience has shown that any attempt by the federal government to impose a uniform 
information system, designed to meet all federal and state needs for educational information, is 
doomed to failure. Users of educational information at all levels of government must decide for 
themselves what information they need and what information they will use. However, the time for 
cooperative development of a national educational information system, designed and maintained by 
a coalition of concerned policy-makers and uiucators at federal and state levels, is now. This view 
is endorsed by the U. S. Department of Education (as evidenced by this NCES Redesign Project), 
by the Council of Chief State School Officers (as evidenced by their June 19, 1985 letter to the 
Administrator of NCES and by their accompanying position paper on the Redesign Project, which 
stated, in part: 

"The Council believes that the National Center for Education Statistics has a vital role in 
responding to educational needs in the following general areas: 

4. Assistance to state and local agencies in the design and operation of activities ct the state 
and local level"), 

and by the Council of State Governments (as evidenced by their 1985 position paper on tne NCES 
Redesign Project, which stated, on pp. 8-9: 
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"What remains [for NCES] is the problem of identifying the best means of providing useful 
statistical information to the [state-level] political decision makers within their unique 
environment. Given that information is 'that which reduces error/ we should conclude that 
providing better, more useable, statistical information to these important political actors will 
encourage improved educational policy decisions ... in the same way that providing better 
information to SEA and LEA leaders has improved their capacity to make better decisions."). 

In an era of federal retrenchment in which the Administration has issued a clear call for less, 
rather than more, involvement by the federal government in areas not clearly within its purview, a 
proposal for federal leadership in the development of an integrated national educational information 
system might appear anachronistic. Yet such a leadership role is clearly within the statutory 
authority of the National Center for Education Statistics. The General Education Provisions Act of 
1974 (Section 406(b), as amended in 20 U.S.C. 1221e-l) defines as a part of the NCES mission: 

"(3) assist State and local education agencies, including State Agencies responsible for 
postsecondary education, in improving and automating their statistical and data collection 
activities." 

A central assumption underlying this proposal for a national educational information system, and 
for a strong federal role in developing and maintaining that system, is that the proposed federal 
leadership position in developing such a system is an essential part of the Congressionally-defined 
responsibility of the U. S. Department of Education. 

This plan for a federal elementary and secondary education data program is based on a set of 
assumptions concerning the willingness of the Department of Education to modify its current 
data-collection activities and to invest the resources necessary to develop an adequate education 
statistics program. In particular, we assume the following: 

(1) All current NCES projects concerned with data on elementary and secondary education 
can be eliminated, replaced, modified,or left untouched to the degree necessary to effect an 
adequate program for collecting, analyzing and reporting data on the condition of education 
in the United States; 

(2) A new program for collecting, analyzing and reporting data on the condition of education 
in the United States need not be constrained by the funds currently allocated to the National 
Center for Education Statistics; 

(3) A new program for collecting, analyzing and reporting data on the condition of education 
in the United States need not be constrained by the present administrative organization of 
the U. S. Department of Education; 

(4) A new program for collecting, analyzing and reporting data on the condition of education 
in the United States need not be constrained by the present administrative organization of 
the National Center for Education Statistics; 

(5) A new program for collecting, analyzing and reporting data on the condition of education 
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in the United States and foreign countries need not be constrained by the present functional 
responsibilities and mission of the National Center for Education Statistics; 
(6) A new program for collecting, analyzing and reporting data on the condition of education 
in the United States should be designed explicitly to meet a range of well-defined needs for 
information on education and schooling; that is, the content and organization of the 
program should be determined by a well-conducted analysis of the needs of data users. 

C. An Expanded Mission for the National Center for Education Statistics 

Many contributors to the NCES Redesign Project view the 1974 statement of the Center's 
mission (defined in the General Education Provisions Act, as cited above) as inadequate to the 
need for such a center, and, as a step backward from the original charter of the Department of 
Education. Although the 1974 mission statement defines a specific set of responsibilities for 
NCES, it does not empower a unitary federal agency with sole authority and responsibility for 
informing the nation on that sector of society labeled "education." As a result, NCES does 
not do enough and other agencies of the federal government, both within and outside the 
Department of Education, do too much in their quest for data on schooling and education in the 
United States. Unnecessary duplication, lack of coordination, and excessive respondent burden are 
well documented in the papers prepared for the NCES Redesign Project. 

The mission we would propose for NCES would make it the federal agency with authority 
and responsibility for collection of data concerning education in the United States. Other agencies 
with specific needs for regulatory data on education (such as the Office for Civil Rights, other 
agencies within the Department of Education, the Department of Labor, the Department of 
Agriculture, and the Department of Defense) might collect such data, provided their activities were 
coordinated by NCES, and only in situations where NCES was unable to meet their programmatic 
needs for such data. 

Our position on an appropriate mission for NCES is consistent in spirit with that advanced 
by the Counci 1 of Chief State School Officers (1985): 

"We strongly urge that the function [of NCES] be a true statistical center that assumes the 
major responsiblity for coordination of the collection, assembly, analysis and dissemination 
for that sector of society under its purview, namely education. 

"The Secretary of Education would be required to make a clear and committed designation 
that the Center would have responsiblity for coordination of statistical data collection and 
analysis activities across the Department of Education regardless of organizational lines 
and/or bureaucracies. This assignment would also require that the Center be charged with 
promoting the integration of the numerous data collection activities conducted by other 
federal agencies (Department of Agriculture, Bureau of the Census, Department of Labor, et 
al.) and related private agencies (National Education Association, American Council on 
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Education, and the testing industry) to minimize burden on respondents and to develop 
increased standardization of terminology. 

"The coordination role would include! 1) first and foremost, the coordination of the various 
activities currently under development in NCES (e.g., CCD, VEDS, NELS-88); 2) 
expansion of the system to include those other data collection activities of the Department of 
Education (e.g., Special Education, Chapter I of ECIA, Chapter JI of the Math and Science 
Act); and finally, 3) establishment of out-reach activities to other agencies to ensure 
appropriate federal and national coordination. Included in this function would be defining a 
common set of data elements across the spectrum, coordinating collection of all statistical 
data, developing efficient collection and dissemination systems (in conjunction with users 
and providers), seeking out current needs for educational information, and providing 
assistance, both technical and financial, to the respondees and users of educational data. 

"Any effort at a ten- year plan, without a clear understanding of the agency's mission and 
philosophy, offers little promise of success. Additionally, in our view, the failure to expand 
the mission and functional boundaries of the National Center to a true center for education 
statistics limits the potential growth to little more than that capacity which exists today." 

Obviously, expansion of the NCES mission to that of a "true" statistical center would require 
commensurate expansion of capability to support such activity. Realistic investments in 
personnel, facilities, equipment, and funding would have to be made. Assumptions concerning the 
willingness of the federal government to provide needed resources were discussed in the preceding 
section. 

In sum, the mission we envision for a federal education data center would embody the 
following goals: 

(1) Coordination of all collection, analysis and reporting of data on education by the executive 
branch of the federal government; 

(2) To the degree possible, collection of data on education required by all other agencies of the 
executive branch of the federal government; 

(3) Meeting the needs of all federal policy-makers for information on the condition and status 
of education in the United States, by developing and maintaining an integrated national 
educational information system; 

(4) Providing leadership and assistance to state governments and local education agencies in 
the development of complementary educational information systems for purposes of 
management and policy analysis; 

(5) Effecting such research, quality control and auditing procedures as are necessary to ensure 
the precision, quality, and integrity of the information provided by a national educational 
information system. 
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Chapter 4 

Designing a New National Educational Information System 

In designing a new data system, one needs to answer several critical questions. These 
include: 

1. What information should be collected? 

This question addresses n^t only the "contents" of individual information elements, but also 
their form, i.e., fundamental linkages between elements that allow or prohibit their use for 
specific purposes. 

2. 1 ow should the information be collected? 

This issue encompasses not only the methods of data collection, but also the categories of 
persons and administrative records which will provide the data. Sample design, timing of 
collection, and provision of standards of data quality-including mechanisms for assessment 
and control-are also key issues. 

3. How should it be made available for use? 

This latter question breaks down to: Who should receive what information, in what form , 
when , and at what cost ? 

These issues address data transmission processes among those responsible for collecting the 
data and maintaining the system as well as information flows linking external users into the 
system. As such they include specification of records to be transmitted, timing and 
frequency of transmission, aggregations and analyses to be performed and reported, 
regulation of access-including timing of data releases, provision of privacy and security 
constraints, availability of micro- versus aggregate records, the costs of access and who 
should bear them. 

From our perspective, an educational information system must be designed to fulfill the 
needs of those who will use the information to enhance the quality of the education experiences for 
which they have authority and responsibility. The fact that parents have such authority and 
responsibility implies that their information needs must be served with the information system. 
The fact that citizens and public officials have such authority and responsibility implies that their 
needs must also be served by that system. And the fact that educators have such authority and 
responsibility as their central concerns implies that their needs must be better met than they are 
currently. 

By addressing whose needs must be served we also inform the issue of what kinds of 
information they should receive. If the information is to serve the needs of those who have the 
responsibility for decisions affecting the quality of education experienced by pupils, then it must 
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allow assessment of the quality of the educational system and the parts of that system which affect 
its quality. Thus, only if we have a conceptual framework for the assessment of educational quality 
can we make appropriate choices about the information to collect and the form in which to make it 
available. 

Such a frameworrw should also lead to a more cohesive system, i.e., an information system 
which allows relevant linkages between its components so that better assessments can be made of 
the effects of changing one part of the system on the other parts. Also, since any information 
system requires allocation of limited resources, a conceptual framework allows one to set priorities 
within the system to achieve maximum benefit from the resources available. 

A. A Conceptual Framework for Describing an Educational System. 

An educational system is an organization which converts resources into educational services 
for pupils. From our perspective, one can specify public education as a system at the level of class, 
school, district, or state. Thes form a nested set of educational systems, with varying and 
changing responsibilities for governance and policy formation. Private educational systems 
typically have fewer organizational levels. 

Ultimately, the success of an educational system-regardless of organizational level-is 
predicated on its outcomes. As a society, we intend the system to help prepare individuals for 
work, for political participation, and for family life. To the extent that education does not play its 
role in preparing students effectively, we desire to improve it. Because of the central role of 
outcomes in evaluating the success of our educational systems, greatly increased efforts have been 
made recently to improve the amount and character of information about pupils' achievements. 
The intent of these changes-at local, state, and national levels-has been to assess the quality of our 
educational systems and to bring about improvements in them. 

An exclusive focus on achievement, however primary as a public signal of the failures and 
successes of a school system, is not sufficiently informative to improve that system. If excellence 
of quality of education is equated solely with high achievement, then we will reward and support 
ineffective schools. Quality of education is not merely quality of outcome, i.e., high achievement. 
Learning, and its educational manifestation-achievement, occur in and are supported by other pans 
of the pupil's life than schooling. Thus, creating an adequate national educational information 
system requires a more comprehensive definition of educational quality and a more systematic 
framework for describing an educational system. 

Children live in widely differing circumstances. Some belong to families where parents 
value and are able to support school learnings, providing time and other resources to supplement 
the school's efforts, and ordering an-' organizing the children's lives to facilitate their 
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achievements. Other parents neither realize the ultimate importance of these learnings, nor have 
they been able, in terms of their resources or skills, to undergird or augment their children's 
schooling. Thus, schools are confronted with wide variations in the educative difficulties they 
face. 

Assessing and improving quality of education requires considering student achievement in 
relation to these difficulties, and evaluating and modifying the educational efforts which are made 
through the process of schooling. To merely focus on achievement means to praise those 
educational units that draw students from educationally advantaged backgrounds, i.e., schools and 
districts of high socio-economic levels; this is a disservice tc those who educate disadvantaged 
students. If we are to improve the quality of education, we need to assess students' educative 
difficulties and search for and detect educative processes which lead to the highest achievements 
desired and possible. 

A conceptual model . In order to apprehend educative processes, we must rely upon a 
conceptual model. This model may be simple or complex and it may be implicit or explicit, but its 
existence is a prerequisite to any understanding of the effectiveness and quality of schooling. Our 
conceptual framework— presented below— for describing an educational system focuses on the 
school because it is at the level of the school that educational activities take place and that pupils 
participate in them. 

By this primary focus we do not mean to imply that all primary educational decisions and 
actions do take place or should take place at the level of the school. Centrally important decisions 
about resources and their allocations, educational goals, and educational activities and programs 
takr -lace at the state level— in governors' offices, legistatures, state departments of education-and 
at the district level as well. We do mean to imply, however, that once these decisions are made and 
contraints are imposed, much of the process of implementation-i.e., using the allocated resources 
to create educational experiences addressing desired outcomes-takes place at the school. Thus any 
framework for constructing a comprehensive description of educational systems must begin with 
the school and expand from it to the other levels of the systems. 

Fundamentally, schools and the communities they serve differ in several important ways: 

♦ Family and Community Environment: 

The families ana communities served by different schools differ in significant ways, .lie 
differ in the resources available in the homes of the pupils for support of theii schooling and 
they differ in types and levels cf aspirations parents have for their children. The family 
composition of the community affects the attitudes, values, and goals of a pupil's peers. All 
of these form the context within which schools can educate their pupils. 
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Schools are faced with differences in levels and types of educative difficulties with which 
their pupils present them. Some present handicaps or limited proficiencies in English. 
Others come with limited levels of prior learning. Thus, pupils who enroll in some schools 
enter with cognitive accomplishments and capabilities, motivations, and out-of-school 
environments and resources whicn make educative efforts easier and less complex than those 
in other schools. 

♦ Resources: 

Schools have available to them different levels of monetary resources and different amounts 
and kinds of non-monetar T resources , such as volunteer time, donated supplies and 
equipment 

These resources are exchanged, allocated, and configured as a teaching staff, facilities, 
educational materials. 
♦ Goals: 

Schools aspire to distinctive goals. For example, some public secondary schools design 
their entire curriculum around post-secondary career paths which primarily begin in selective 
colleges and universities, while other schools, e.g., "vocational" ones, may focus their 
whole program around immediate job entry to skilled and semi-skilled occupations. 
♦Process: 

Schools offer educative experiences for which they require or encourage pupils' 
participation. These include work experience, homework, and extra-curricular activities as 
well as in-class experiences. Schools also structure these experiences with different 
standards. These standards influence the pursuit cf goals with different expectations for 
performance, differing time allowances for accomplishment, and differing criteria for 
selection into subsequent experiences. 

Schools also differ in the types and amounts of participation of their pupils in these 
educative experiences as well as in the range of experiences made available. These variations 
include differences in selection, participation, and completion of educational programs, 
course work, and homework as well as differential school attendence. 
♦ Outcomes: 

All through the schooling process, to the conclusion of secondary schooling and beyond, 
schools differ greatly in the goal-relevant accomplishments and achievement of their pupils. 
These include cognitive capabilities, credentials, and career and life paths generally. 
To understand schooling in ways that carry meaning for those who participate in it and are 
concerned about it and its consequences, none of these areas of distinctiveness can be 
glected. School outcomes may differ by intent as well as by the efficacy of programs and 
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activities. Schools, school districts, and entire school systems are presented with considerable 
variations in the levels of preparation, handicaps, and other educative difficulties that their pupils 
bring to the schooling process and these have profund consequences for outcomes. And schoois 
and the larger systems within which they are embedded really £o differ in their effectiveness. Thus 
it is vital 

•to describe, against a coherent conceptual frame, each of these differences in a cohesive 
fashion, as well as 

•to attempt to sort out the reasons for differential outcomes against the structure of their 
origins. 

Figure 1 displays such a conceptual framework. It focuses on the schooling process, 
distinguishing teaching activities from pupils' exposure and participation in the resulting educative 
activities. And it traces these aspects of the process to their origins: prior and contemporaneous 
characteristics of pupils, community and family expectations, curricular goals, and resources, as 
well as linking them to their consequences. 

The intent of this figure is to clarify that educative difficulties and goals must be accounted in 
order to assess the quality of a school system or an individual school. Since some schools must 
exert greater and more costly efforts in increasing pupil participation than others, the factors which 
influence the degree of effort required must be accounted if that effort and the resulting "quality" of 
the educative process are to be adequately diagnosed. Similarly, the desired outcomes or goals of 
the educational system affect the kinds of educational experiences offered pupils. Thus, schools 
which focus solely on nanDw or atypical goals may produce learning outcomes which either 
advance tested achievements relative to non-tested ones or seriously skew them. Much of this 
effect will depend on whether the test content is broadly defined, balanced, and clearly articulated to 
legitimate educational goals. 

What is a good school system ? Within the context of school quality assessment and its 
bearing on school improvement, thi< framework treats goals, educative difficulties, and resources 
as pre-conditions (or background elements) for process description and outcome interpretations. 
We may then explore the implications of this framework for meaningful and valid comparisons of 
schools and school systems. 

As we outlined above, educational systems are faced with difficulties in educating their 
pupils and these difficulties vary greatly in type and degree. Additionally, they have distinctive 
goals for their pupils and these goals are related to the characteristics of the pupils which they 
serve. In order to compare systems with respect to the quality of the education they provide, it is 
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Figure 1, A Conceptual Frame for the Schooling Process 
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extremely important to account both the educative difficulties ot their students and the goals the 
schools have for them, i.e., variations in type of educational program are strongly conditioned by 
the prerequisites of the pupils served and ^ * goals which the schools wish to attain for these 
pupils. Goal variation becomes increasingly important in the later stages of the schooling process. 
Thus, secondary schools respond with distinctive track selection ratios, programs, and course 
offerings to satisfy community and family expectations. Elementary schools, on the other hand, 
have much more commonality in goals, at least within fundamental or "basic" educational skill 
areas. Thus, in the earlier stages of schooling, the primary problem is accounting for the educative 
difficulties schools face rather than focusing strongly on differences in aspired outcomes. 

Resources are, of course, mandatory to mount any educational program. And although the 
resource levels needed to mount programs of adequacy or quality are widely debated, the fact that 
level of resources influences educational effectiveness is not at issue. Resources are used to mount 
the educational process, in order to attain the goals while dealing with the difficulties. Clear 1 */, 
given the same goals and the same difficulties, organizations with different resource levels are not 
able to offer the same experiences. 

How is school effectiveness and quality of schooling revealed in the achievement of pupils? 
Quality of scV^oling resides in the process of schooling. Schools differ in the educative 
experiences they provide and in their success in gaining pupils' participation in them. These 
experiences are the services that a school provides to pupils and their parents. A good school is 
one which provides effecrive learning experiences or services actually leading to set goals and 
which accomplishes pupil participation in hose educational activities. The quality of these 
educational activities in which pupils participate is then reflected in pupils' achievements. But the 
knowledge and skill a pupil starts with will limit the outcomes of even the most effective schooling. 
And pupils who do not bring to school attentiveness, perserverence, ana capacities to rapidly digest 
the instruction they receive will gain less from any sc nool experience than those who do bring 
them. Thus, two educational systems with similar goals, but different achievement levels can only 
be assumed to differ in quality if they also face similar educative difficulties with their pupils. 

What are the implications of this framework for the design of a new national educational 
information system? First-and foremost-in order to validly assess the quality of our educational 
systems, the data ollected must be comprehensive. That is, these data must measure the 
background conditions and the current difficulties pupils present to t'^eir schools, and they must 
assess the goals pursued by the schools and the resources available to address them. They must 
record the experiences moi nted-i.e., the educational services offered to and required of 
pupils-with these resources and they must at^um pupils' participation in these experiences and 
the outcomes resulting. 

Seconu, in order to help improve tue quality of education received by pupik, the linkages 



9 

ERJC 



51 

5o 



among these aspects of schooling must be revealed. For example, we need better information 
about how resources are used to mount educational experiences, i.e., what are the abilities and 
expertises of the teachers who conduct these experiences? What are they paid? How are 
educational activities and experiences created out of facilities, materials, and the time and effort of 
educators? We also need better information on participation. How are pupils with differing 
backgrounds and characteristics selected for participation in particular experiences? What are the 
actual rates of participation of various types of pupils in these experiences? Finally, we need clues 
to the effectiveness of tracks, programs, and experiences-especially for pupils with differing 
characteristics. What are the outcomes--in both the short- and long-runs--of these pupils' 
participations? 

B, Needs and Issues Addressed bv an Integrated Information System: Some 
Examples 

The current national data system in elementary and secondary education— as we discussed in 
Chapter 5 above— is piecemeal and fragmented. We have very little information aboi the family 
backgrounds and ducative difficulties and characteristics of pupils and the little ve have is 
collected sporadically and exists in data sets which are iarely linked to resources, participation, and 
outcomes. We have basic information on revenues, expenditures by accounting category, and 
educational personnel. But these data are not linked together in ways which allow the tracing of 
resources flows. Outcome data are extensive in some data bases (e.g., the National Assessment of 
Educational Progress and state assessment programs), but these data are not linked to data about 
program offerings or participation in any cohesive fashion. Participation data (e.g., from the 
National Education Longitudinal Studies, or the Survey of Income and Program Participation) are 
nc: effectively linked to institutional data on expeditures or personnel. 

The virtue of an imegrated national educational information system built around a 
comprehensive conceptual framework lies in guaranteeing that the central priorities for data are met. 
This framework helps deliniate the data content to be include md the setting of priorities 
concerning that content. However, some of the irpst important aspects of a new data system relate 
to its ability to address the new needs discussed i Chapter 2. These needs primarily relate to using 
a new information >ystem to address service delivery, and the linkage of delivered sevices to 
resources, pupils, and outcomes. From this perspective it becomes clear that the linkages between 
data elements are just as important as the existence of particular elements and the degree to which 
they comprehensively and validly represent a particular aspect of the educational system. In the 
examples below, we illustrate some of these linkage issues. 
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Quality and effectiveness of educational systems, Above we made the point mat 
achievement, although central to the assessment of educational quality is not sufficient. First, one 
cannot adequately evaluate achievement as an index of effectiveness without knowing to which 
goals the system is directed- It is the discrepancy between desired outcomes-goals-and actual 
outcomes which diagnoses the effectiveness of the system. To the extent that systems are directed 
to different goals or have different distributions of planned outcomes, they cannot be validly 
evaluated without knowledge of their goals. In addition, even such holistic evaluations of quality 
must take into account the educative difficulties which the system faces. For example, schools, 
districts, and even state educational systems serve pupils of widely varying lirguistic backgrounds, 
parental supports and resources. These discrepancies will lead to different outcomes even when 
system goals and resources are similar. This implies that goals, pupil characteristics-backgrounds, 
difficulties~and resources must be linked to each other and to outcomes in order to assess quality 
and effectiveness. 

Also, if we are to probe quality and effectiveness so as to give guidance for change, these 
linkages must also be made at the microl evel, i.e., the record structure must allow resources to be 
linked to personnel, personnel to be linked to services, and services to be linked to pupils and their 
achievements. 

Resource flows . One of the centrd data needs not currently met by the existing data system 
is resource flow information. The fragmented data collection system produces information on 
revenue sources, expenditure categories, and personnel, but these information components are not 
linked in such a way that one can tell which monies are spent on what. A part of the "problem" is 
in the fund accounting system of most school districts. However, the major issue resides in the 
lack of linkage between the accounting categories used for expenditure reporting and their flack of) 
articulation with personnel categories reported. A system must be designed to allow micro linkages 
of accounting system expenditures to personnel records und employee characteristics. 
Additionally, these records and characteristics must be linked to the experiences and services 
offered in the district and (ultimately, see below) to pupils. 

Pupil participation The fundamental requirement for assessing both the equity and 
effectiveness of educational programs is information on who participates. Equity, at its root, has to 
do with social and legal determinations of the amounts of resources and the kinds of programs with 
should be mounted for particular categories of pupils to meet their needs and to fulfill society's 
icsponsibilLles to them. The data problem is made more difficult by the fact that educational 
agencies mount quite different programs, requiring different amounts of resources, and resulting in 
differential participation, by pupils of different types. Fundamentally, one would like to Know 
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•the quantity of resources and the characteristics of programs and experiences participated in 
by various kinds of pupils; and 

•the outcomes of pupil participation by both pi^il and program or service characteristics. 

These kinds of relationships require linkages of service characterisics with pupil characteristics via 
participation information. These linkages, in turn, require microdata on individuals. 

Productivity and efficiency . Quality and effectiveness are not the same as productivity or 
efficiency. The former terms refer to outcomes of a system in relation to its gzzls and the 
difficulties it faces. Productivity and efficiency relate quality and effectiveness to the resources 
used to mount educational efforts. To the extent that one educational system is of the same quality 
or effectiveness as another, but has used fewer resources to accomplish this, it is more productive 
or efficient. Thus, to evaluate productivity or efficiency one needs to link resource information to 
outcomes, via goals, difficulties, services and participation. In this sense, the micro record 
structure and data collection needed to support analyses of educational productivity and efficiency 
are the most stringent of all. They require information on all three of the above topics: quality and 
effectiveness, resource flows, and pupil participation. 

C. Some Benefi ts of a New National Educational Information System. 

The consumers of information about educational systems include parents concerned about the 
education of their children, citizens worried about the quality and efficiency of the education their 
tax dollars finance, professional educators making decisions about programs and pupils, and public 
officials desiring to design laws, requirements, and resource allocations which will effectively 
improve education. All of these consumers are concerned that the information which reaches them 
be relevant and useful to their needs, timely, and accurate. 

Common to all of the consumers are concerns about quality and effectiveness. It is this 
information which is most desired in the public debate over education. Parents want to know about 
the quality of education their children receive and about the qualities of the educational alternatives 
available to them. Citizens and public officials wish valid assessments of efficiency to know that 
resource allocations are wisely made and carried through desired outcomes. 

Resource flows are important information for public officials in making determinations of 
how much and how to allocate resources. Federal officials have special concern foi hov/ federal 
resources are channeled to pupils and the impact of these resources on pupils with specific 
characteristics. State officials, in fulfilling their responsibilities, have been modifying state 
educational systems in ways that require comprehensive information about participation in 
programs, courses, and other services, standards of performance and actual outcomes. Local 
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officials are newly concerned that they are effectively monitoring service delivery, participation and 
achievement 

An effectively integrated system-incorporating the microdata and records necessary to 
produce these new types of information-is needed by all concerned parties. The benefits of ? 
cohesive system of this type producing national and state comparable data would be far reaching. 
Not only would the majority of consumers of educational information be provided with relevant, 
integrated, timely, and accurate information at these two levels, but the establishment of such a 
system would produce similar changes in district-level information systems. This, in turn, would 
increase the comprehensiveness and comparability of the information about education taking place 
in local communities. Thus, the national information system, as it is established at state and 
national levels will introduce cohesion in the total system. 
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Part II. The Design and Implementation 

of a 

New National Data System 
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Chapter 5 
Alternative Designs for 
A National Educational Information System 



A. Introduction 

What is a National Eductional Information System? In the previous chapter, we discussed at 
length the need for an integrated national educatiional information system built around a 
comprehensive conceptual model of the schooling process. In the development of such a system, 
three questions need to be addressed What sort of data base is called for? What processes will be 
used to get information into the data base? What processes will be used to get information out of 
the data base? In this Part of Chapter V, we offer general answers to these three questions and, 
thereby, describe in broad terms our view of what a National Educational Information System must 
be. 

The Data Base. In order to meet the information needs of the broad array of local, state, and 
national educational decision makers identified in previous chapters, the data base must be 
structured to provide information on all aspects of the schooling process as described in our 
conceptual model. This means that the data base must be comprehensive; put simply, it must be 
adequate in scope and coverage; it must contain accurate, appropriate, and timely information on (1) 
the school setting, (2) the schooling process itself, and (3) the outcomes of schooling. 

Data on the school setting must include information on the environmental factors that impinge 
on the school, such as community and t imily characteristics and expectations. School setting data 
also must provide information on financial revenues as well as other incoming non-monetary 
resources available to the school. The data base also must include information on the educative 
difficulties which face the school, such as pupils' capabilities, motivations, handicaps, English 
language facility, and out«of-school supports. 

Data on the schooling process must include a broad range of information. The data base 
must provide information on the educative goals of the school, its objectives, and curriculum. It 
must prov'le information on allocated resources-facilities, staff, equipment, materials, and other 
non-monetary resources made available for school use. It must provide information on educational 
pursuits, that is, curricular offerings, standards, teaching-and school-related activities. Equally 
important, the data base must provide information on the extent of pupil and parent participation in 
the process of schooling. 

Finally, the data base for a national educational information system must include information 
on the outcomes of schooling, incuding pupil achievement data as well as information on such 
outcomes as high school graduation, drop-outs, political participation, employment, and 
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post-secondary matriculation 

A second major requirement of the data oase, in addition to being comprehensive, is that it be 
integrated, thai is, that its data elements, files, and records be linked to one another. The user must 
be able to ask and have answered questions about the relationships among background 
characteristics, the schooling process itself, and the outcomes of the process. The data base must 
be able to provide information to answer such questions as, "What dollars buy what services for 
which students with what results?" Or, "What programs staffed by what types of teachers are 
effective for pupils with particular educative difficulties at what costs?" Only if the data base is so 
structured as to allow relevant linkages among its elements, files, and records will the requirement 
for an integrated educational information system be met. 

Getting information into the data base . The dual requirements for a comprehensive and an 
integrated system demand, in turn, that data be collected in micro record form, as opposed to 
macro-record or aggregated form. We define a micro record as a datum on an individual person or 
entity rather than a datum on a collection or aggregate of individual persons or entities. A micro 
record can be dealt with as an individual datum or aggregated; for example, individual micro 
records on pupils can be aggregated to the school level. A macro-record, on the other hand, 
generally cannot be disaggregated. More importantly, the micro record permits of linkages with 
other micro records; for example, micro records on individual pupils can be linked w'di micro 
records on individual pupils can be linked with micro records on individual teachers and, in turn, 
with micro records on specific curricular offerings in which the teachers and pupils are 
participating. The micro record formpt, through its linkage capability, permits the information user 
to ask questions about relationships among the sets that make up the data base. Thus, the major 
requirement to be met in designing a process for getting information into the data base we have 
described above is that data be collected and stored in micro record format 

Getting information out of the data base. In the previous two sections, we identified the 
major requirements that must be met in establishing the data base and in getting information into the 
data base. A third question remains, namely, what processes will be used for getting information 
out of the data base? 

First, a national educational information system must be able to deliver information of a 
comprehensive and integrated nature on the schooling process in the Nation as a whole, that is it 
must be capable of delivering information that is nationally representative. It must be able to report 
on the status and progress of elementary and secondary schooling in the United States. It also must 
be able to deliver information on sub-national or regional populations. In addition, we have taken 
as a given that the system must be capable of producing information that can be used to compare the 
condition and progress of education in the various states; in short, the system also must be capable 
of delivering information that is representative of each of the fifty states. 
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While such a requirement dictates attention to how information gets into the data base, e.g., 
the sampling designs which will be employed, also dictates- along with the ^eviously identified 
requirements of comprehensiveness, integration, and micro record formats- what types of reports 
must be available to users of the system. Users, with the possible exception of researchers, 
generally will not be interested in micro records per se but rather reports developed from the 
processing— e.g., tabulation, aggregation, and analyses— of micro records. Thus, while micro 
records represent the form in which information flows into the data base, reports based on 
processing of the micro records generally represent the form in which information flows out of the 
data base. Yet, a simple proliferation of reports will not meet the needs of the broad array of local, 
state, and national decision makers which we have identified in previous chapters. A national 
educational information system must be capable of carefully tailoring its reporting formats and 
mechanisms if it is to serve the particular needs of this broad array of decision makers. Certain 
decision makers, for example governors, have n^eds for only certain kinds of information and not 
for other kinds; the system must be capable of meeting these needs. In short, the system must be 
capable of screening and matching its reporting formats with the needs of particular users. In 
addition to questions of content, the screening and matching require attention to establishing the 
mechanisms necessary to actually get the reports to decision makers and decision makers 10 the 
reports, and in the case of researchers, to the relevant portions of the data base itself. 

Finally, the process for getting information out of the system has to pay serious attention to 
timing. Unless the information is available when needed, the content and form of the reporting 
mechanism makes little difference. Timing involves setting priorities for reporting different sets of 
information to different users, as well as priorities for providing different users access to different 
sets of information. In summary, a national educatir nal information system must be capable of 
delivering periodic and differentiated reports on th status and progress of schooling to a broad 
array of local, state, and national decision make as well as making available to different users, 
including researchers, special reports on arid put . c use samples relevant to particular aspects of 
elementary and secondary schooling in the Vir :l States and in the several states. 

By what criteria should we judge natir ucational information system? In Part 1 above, 
we described in broad terms our views of ' .ational educational information system must be. 

In our description, we identified a number ~u acquirements that such a system must meet. In this 
part, we briefly reiterate those requirements as well as certain additional requirements, and identify 
them as the basic criteria that we believe should be used in judging any system, present or future, 
that purports to be a national educational formation system. Our basic criteria are as follows: 

1. COMPREHENSIVENESS-the system must have a data base capable of providing 
information on all pertinent aspects of elementary and secondary schooling including 
the school setting, the schooling process itself, and the outcomes of schooling. 
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2. 



INTEGRATION- the elements, files, and records in the data base must be linked; 
all data sets must be capable of being related to one another. 



3. micro record F ORMAT-all data must be collected and stored in micro record 
format, with a micro record being defined as a datum on an individual person or an 
individual entity. 

4. REPRESENTATIVENESS- in addition to being nationally representative, the 
information in the data base must be representive of each of the fifty states, as well 
as representative of other important variables such as sex, racial-ethnic composition, 
urbanization, and so on. 

5. ACCURACY-all data must be verifiably accurate ; they must be subjected to 
rigorous quality control procedures including audits, reinterviews as a routine part of 
data collection, controls on data entry and data processing, consistency and 
completeness edits, and regular and routine calculation of measures of variance. 

6. COMPARABILITY-data from different jurisdictions must reflect the same concepts 
and definitions; common units of reporting and common definitions are necessary 
precursors of useful data aggregations. 

7. TIMELINES S-in general, data must be limited to that which can be collected, 
stored, and analyzed within three months and reported to policy makers within the 
year. 

8. PRIVACY AND SECURITY- because some of the elements, records, and files 
contain information about individuals, e.g., personal identifiers necessary for 
longitudinal studies, strict confidentiality and security measures must be in force. 

9. PROCESSING AND ANALYSlS-a specific schema must be available for 
processing the micro records in a manner designed to optimize the analytic capacity 
of the system. 

10. INFORMATION FLOWS-the system must be capable of screening and matching 
its reports to meet the particular needs of particular users; a wide array of reporting 
formats and access mechanisms must be available to serve the different users; 
specific priorities must be set for meeting the different timelines imposed by the 
needs of different users. 

1 1 . COSTS OF TRANSMISSION/ACCESS- a pattern of shared user costs should 
characterize the system; rather than rely exclusively on federal support for 
transmitting information to users and/or providing them access to information, a 
national educational data system should also draw support from a program of user 
fees and thereby increase its capacity to serve the differing needs of its users; equally 
important, transmission/access modes should incorporate the latest developments in 
electronic communications technology. 

How does the existing system stand up against these criteria? In this section, we assess 
current NCES data-collection activities against the criteria identified above. In Chapter 2, we 
discussed these activities at some length, describing them as an unarticulated set of projects rather 
than as a program or system for providing information on the elementary and secondary schools of 
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the nation. As such, they fail to meet the criterion of INTEGRATION. They exist as a discrete set 
of activities. In their pres' nt forms, it would be extremely difficult if not impossible to merge these 
into an integrated system, principally because they also fail to meet a second criterion, namely, that 
all information in the data base be collected and stored in micro record FORMAT. 

Even if it were possible (which we do not believe it is) to integrate the current activities, 
such a system would still fail to meet the criterion of COMPREHENSIVENESS. As we pointed 
out in Chapter 2, and as was amply documented by the author: of papers for the Redesign Project, 
current NCES data on elementary and secondary education are simply inadequate in scope and 
coverage. Data are lacking on substantial aspects of elementary and secondary education, for 
example, the type and quality of instructional activities in our nation's schools. 

The present set of activities also fail to meet the fourth criterion, 
REPRESENTATIVENESS. Most of the data collection activities of NCES support estimates of 
parameters only for nationwide, or occasionally for regional populations; for the most part, data 
representative of each of the fifty states are not available. The ACCURACY of current NCES data 
is also in question. The authors of the papers for the Redesign Project provide extensive 
documentation regarding the inaccuracy and impreciseness of NCES data. 

Nor is the criterion of COMPARABILITY met. Common units of reporting and common 
definitions, particularly in the case of the Common Core of Data, often fail to exist, thus producing 
substantial differences in the meaning of data reported by the different agencies. As we also 
pointed out in Chapter 2, current data-collection activities fail to meet the criterion of 
TIMELINESS. In several instances, the most recent data available are two, three, four, five, and 
even ten years old. 

The one criterion of the eleven we have identified which current NCES activities apparently 
do meet is PRIVACY AND SECURITY. There was little or no evidence in the same 60 odd 
papers that undergird the synthesis report that privacy and security of current activities was a 
problem; most comments on privacy and security related to proposed new modes of collection, 
storage and dissemination. 

The remaining three criteria that we have identified-PROCESSING AND ANALYSIS, 
INFORMATION FLOWS, and COST OF TRANSMISSION/ACCESS-are perhaps more 
germane to the new system we propose below. Therefore, while it is true, it may be unfair to say 
that the present set of NCES data-collection projects fail to meet these criteria. 

Our answer, then, to the question of how the existing system stands up against the criteria 
which we have identified is: "NOT VERY WELL, AT ALL!" Our assessment of current NCES 
data-collection activities against the criteria we have identified leads us to the conclusion that 
TINKERING WITH THE PRESENT "SYSTEM" IS NOT THE ANSWER. WHAT IS NEEDED 
IS A RADICALLY IMPROVED, AND VASTLY MODIFIED, NATIONAL ELEMENTARY AND 
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SECONDARY EDUCATION DATA SYSTEM-ONE THAT MEETS THE ELEVEN BASIC 
CRITERIA WE HAVE IDENTIFIED. We turn now to a discussion and description of the 

B. The Use of Educational Information 

A conception of use . Earlier in this report, we distinguished between managerial and policy 
uses of information. Managerial uses of information primarily relate to ongoing decisions 
necessary for an organization to carry out its mission. The nature of these decisions is predicated 
on the established mission and specific goals of the organization, and the specific organizational 
structure created to carry these out Policy decisions relate to modifications in the mission 01 the 
specific goals of the organization, and to changes in its organizational structure. 

Managerial uses of information occur totally within the educational system. Policy uses 
occur both within and outside of the formal boundaries of the system. Managerial uses occur at 
both the local and the state levels. Local districts have local record systems and information flows 
which inform decisions of teachers, counselors, principals, central administrative staff, parents, 
and students. These information systems support decisions about hiring and purchasing, 
curriculum and instructional planning, and pupil participation in programs and activities. State 
educational agencies--as Ascribed above-have records on local district and school activities which 
support decisions about funding flows and sanctions for non-conformity with state laws and 
regulations. 

Policy uses are not restricted to the educational system itself. Public discussion of 
educational issues and of system quality and performance occur at both the local and state levels. 
Individual citizens and interested groups participate in these discussions and much of the 
information flow occurs through news media. At the local level, the ultimate foci of these 
discussions are decisions by the governing boards. At the state level, the foci are decisions of 
legislatures, governors, chief state school officers and state boards of education. Much of the 
information informing these discussions and the decisions flowing from them comes from the 
information collected and recorded for the purpose of managerial decision making. Special 
analyses and reports are prepared from these data bases to enlighten discussion and decision. 
Increasingly however, these managerial data are supplemented by data collections not intended to 
support managerial decisions. Many states are collecting data-e.g., test performance 
information— which are not used for regulatory purposes but are intended solely to inform debate 
and policy formulation. In addition, federal micro records-from the High School and Beyond 
survey-have been used to produce special tabulations for policy purposes in several states. 
Recently, the National Assessment of Educational Progress has also allowed states to pay for 
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sample supplementations for similar purposes. 

As we have discussed earlier in this report, the federal role in this information system is 
singularly important. There is a longstanding federai responsibility for general reports bearing on 
the condition of education in the United States, and for specific data collections and analyses 
supporting the formulation of federal education policy. 

However, the current condition of education has added to this responsibility. As we noted in 
Chapter I of this report, the rapid and profound changes in the structure and a participation in 
educational decision making stimulated by the reform movement have generated entirely new needs 
for the collection, analysis, and distribution of educational information. There are now strong calls 
for the federal component of the nation's educational statistics system to produce and disseminate 
entirely new kinds of information and to make this information available-on an accurate and timely 
basis--so that state educational systems can be validly compared and state decision makers can be 
provided with quality information to support productive state policy formulation. 

In the rest of this subsection we will lay out a framework for the decision processes at the 
state and local levels and draw from this framework the nature of the data that need to be collected, 
analyzed, and reported to fulfill federal responsibilities to the public and to the states. We will also 
address the issue of how these data should be made available for use by those who must make 
important decisions about our educational systems. 

Schooling decision processes . In our conceptual frame for the schooling process (Chapter 
4), we set forth schema for characterizing schooling, together with its preconditions and its 
outcomes. What was not explicit in that schema were the kinds of decisions which must be made 
in order to carry out schooling. It is these decisions which require information. And it is this 
information which must be collected, organized, and distributed before it can contribute to these 
decisions. We will first analyze the decisions made at the local level, mentioning some of the 
constraints which narrow decision options. We will then look at the state policy role, exploring the 
nature and methods by which states influence local education decisions. This will then allow us to 
discuss the kinds of data needed by particular decision makers and the form that it should take. 
Finally, we will outline some of the policy options for information distribution. 

Decision processes in local education agencies have many participants- school board 
members, administrators, teachers, parents and students. If we follow the schema of Figure 1 
(Chapter 4), resomces flow into the school, are converted to staff, facilities, equipment, and 
m n ^rials, which-in turn-are converted into educational pursuits. Pupils participate in these 
pursuits and, as a consequence, learn, accomplish, and achieve. Each stage of this conversion 
process- (1) monetary to (2) human and material resources to (3) offerings and activities to (4) 
pupil participation-invol /es fundamental educational decisions. Th ,se decisions constitute the 
schooling process within the constraints imposed by existing policies and available resources. 
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To fully create the schooling process, four basic types of decisions must be made: (1) 
budgetary, (2) hiring/purchasing, (3) curriculum/instructional, and (4) participation . These are 
illustrated in Figure 2. 

Budgeting decisions allocate incoming resources-primarily monetary-to budget categoric" 
These allocations are clearly constrained by the total resources available from all sources. The 
decisions are made by governing boards on the basis of information and recommendations provided 
primarily by ackninistrators and they incorporate the schooling goals held by the participants. 

Hiring/purchasing decisions are made by administrators using the budgeted resources. The 
purpose of the budget is to formally constrain these decisions to follow the goals and priorities of the 
budget makers. Stated policies (formalized goals) guide administrators in making decisions within 
budgetary guidelines. Also, the availability and prices of personnel and products with particular 
characteristics limits these hiring and purchase decisions. 

Curriculum and instructional planning take place within the limits imposed by employed 
^rsonnel and the facilities, equipment, and materials purchased by the local agency. These 
planning decisions further implement school goals insofar as they are internalized by the personnel 
making them or are codified into formal policies. The planning creates educational 
activities— programs, courses, units, etc.-and organizes, staffs, provides facilities and materials for, 
and sequences them. 

Participation decisions are constrained by the available offerings. If a program, course, or 
unit is not offered, a pupil cannot participate in it. Participation decisions are made by teachers, 
counselors, administrators, parents, and pupils. Many of these decisions are jointly arrived at, but 
all are made subject to school policies about eligibility for participation. Most of the eligibility 
policies have formal elements based on past or present pupil characteristics such as grades, test 
scores, legally structured definitions of linguistic proficiency, handicap, or prerequisite experiences. 

Finally, these experiences, together with what the pupils bring to them, produce learning and 
achievements, which precurse the subsequent outcomes of schooling. 

By using this schematic portrait of local schooling decision processes, we do not mean to 
imply a linear chronological and causal order. Clearly, for example, some curriculum decisions can 
be made before the hiring decisions necessary to implement them are made and some hiring may take 
place before budgets are formally approved. We do imply, however, a logical order. Tuis logical 
order permits us to trace the flow of resources as they are converted into pupil experiences and thus 
to analyse the decisions which produce these conversions. The evidence produced at the local level 
to inform these decisions thus forms the local information base which can be used to create data for 
state and national information systems. Also, as the decision processes are constrained by 
policies-formulated at both the local and state levels-a conceptualization of the kinds of decisions 
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and their interrelations helps to define both the content of potential policies and the kinds of 
information useful in formulating them, i.e., it provides criteria for a national educational 
information system. 

Information to inform educational policy . We now turn to state policy making. The purpose 
of state educational policies is to influence the actual experiences of pupils and via these experiences 
to affect the learning, achievements, and subsequent life paths of these pupils. The only way in 
which such policies can have these effects is by influencing the decisions of local educators. These 
policies-state laws, regulations, formal guidelines-generally provide resources, contingent on 
specified events or occurences, or require or prohibit local agency actions. They may also attempt 
to modify the context within which decisions are made. In any case, they constrain or attempt to 
consr in local decisions. 

Li Figure 3 we exhibit some of the effects of state policies/laws on local decision making. 
The left part of the figure~the row headings-lists some of the decision contexts and decisions 
displayed in Figure 2. Thd column headings list some common types of state laws and policies 
The entries corresponding to a paired row and column indicate whether particular policy or law 
constrains a specific type of decision or modifies a decision context Thus, for example, 
certification rules consxain local hiring decisions and might also affect the pool of individuals 
considering teaching careers. Another example, might be the effect of new graduation requirements 
on pupil decisions about course taking, school decisions on course offerings and teacher hiring, and 
budgetary allocations. 

These examples also indicate some of the primary data needs of those who formulate and 
influence educational policy at the state level. At the minimum, information is needed which not 
only characterizes the outcomes of local decisions, i.e., information about the budget, about the 
human and material resources hired and purchased, about the educational programs and activities 
offered, and about participation. Information is also needed about how these outcomes were 
produced, i.e., how monetary resources were converted f o human and material resources by 
expending salaries and purchasing products, how these resources produced courses, programs and 
activities by consuming materials and the time of educational peisonnel with particular skills and 
characteristics, and which pupils participated in which activities. 

This framework focuses on two essentials. (1) the decisions made by parents, pupils, 
teachers, and other professional educators about the conduct of and participation in actual 
educational experiences; and (2) the constraints imposed by policies-state and local-on these 
decisions. If the policies are to be effectively formulated, the knowledge base about the decisions 
must be relevant, comprehensive, and accurate. This perspective forms the basis for the design of a 
national data base. 

We nou here that an information base intended for regulatory or managerial purposes overlaps 
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but is not identical in concept to one focusing on policy. A managerial information system might be 
less comprehensive in content if existing state policies address only limited aspects of educational 
decisions. But, since state laws and regulations affect each school district, it is mandatory that 
equivalent information be collected for all of the state's districts if conformity to law and policy is to 
be monitored. Thus, policy deliberation on some topics might require more extensive information 
on a smaller number of educational units than management or regulation. Similarly, federal 
education policy relates to the same local educational processes and decisions thai local and state 
policies address, even if the modes of impact are distinct. We believe that this implies that the 
conceptual framework for the information to be collected is compatible for state, local, and federal 
policy information as well as for state management information. The fundamental differences have 
more to do with information priorities rather that with the basis for specifying relevant types of 
information. 

Thus, we believe this schema provides us with some of the necessary characteristics of a 
national data base that can be used to productively inform debate and decision about improving 
education. 



C. The National Data Rase 



As we discussed above, a statistical information system has three aspects: a data base, 
processes for the collection and entry of information into the base, and processes for transforming 
and transmitting information out of the base. In the previous subsection, we discussed the kinds 
and levels of use of educational information for managerial and policy decisions. From this point 
we will proceed to draw on the implications of this discussion for the content and structure of the 
data base. Subsequently, we will explore some of the alternatives for getting this information to 
those who can use it for their decisions and then discuss options for data collection and data base 
creation. 

A conception of the data base . The core of a data base is its content and organizational 
structure. The content of a data base consists of the definitions of the data elements included. The 
structure of the base consists of the records containing these elements, the files containing these 
records, and the linkages among them. 

Data elements characterize specific entities and the elements which characterize a single such 
entity are often kept together. These collections are teimed records. For example, pupils might be 
characterized by features of tneir home environments, the courses they have taken, or their tested 
achievement. Similarly, teachers c^uld be characterized by their credentials or their employment and 
salary histories. Districts, schools, educational activities, and equipment could all be entities for 
which files are constructed. Local education agencies maintain files and records with many of these 



kinds of entities as the focal unit. 

Thus, an accounting system is a collection of records of transactions. An employee file 
contains records for each employee. A district level school file contains records on each school in 
the district. A high school catrlogue or student handbook contains "records" on each course 
offered And a transcript file contains records for each pupil in the school. We term these "basic" 
files: micro records . 

Up to this point in time, the statistical information in the Common Core of Data has not 
included micro records for pupils, for educational personnel, or for school programs or activities. 
Other data bases do include micro records. Examples are the pupil records of High School and 
Beyond and the National Assessment of Educational Progress. The Common Core of Data consists 
primarily of summary or aggregate records. Thus school districts count pupil and employee records 
to create enrollment figures and personnel distributions and these new summary records are further 
summarized by the states. Similarly, accounting transactions are summarized into ledgers and 
financial reports which are transferred to summary records for transmittal to state and federal data 
bases. Thus, for example, the school district records in the Common Core of Data contain some 
school district level micro data— e.g., identification number, address, fiscal and control status, type 
code, etc.-but the primary data contained in these records are summaries of pupil, employee, and 
school micro records maintained at the school or district site. 

One primary recommendation of this report is that the federal data base contain micro records 
for pupils, personnel and material resources, and educational activities. It is not advocated that the 
federal collection of these mitTO records constitute censuses of pupils, ersonnel, and activities. 
Although, within the data collection alternatives outlined below, sc.: \* states may wish to explore 
this option in the reconfiguration of their own management and policy information systems. Thus, 
the structure of the data base being proposed is an integrated collection of sampled micro records. 

Such a data base structure is mandatory if the process of conversion of resources into 
experiences outlined in the previous subsection is to be traced. The critical policy decisions now 
being made at the state level are specifically intended to influence and constrain the critical decisions 
in this conversion chain. It is this chain of decisions which is the key to the improvement of 
educational quality and which constitutes the focus of the new educational reforms. Only micro 
records for personnel, pupils, and activities can produce the kinds of analyses and reports necessary 
to inform and evaluate these new policy initiatives. 

For example, linkage of course taking to achievement-in the sense of tabulating the 
achievement test results for students with different patterns of course taking-is impossible without 
student micro records containing both course taking and achievement data. Separate school, district, 
or state aggregates of course taking and achievement do not permit such reports. Similarly, 
tabulation of the qualifications of the teachers who teach specific kinds of courses is impossible 
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without integrated micro records linking teachers and their qualifications to the particular courses 
they teach. 

File structure and content . In the rest of this section we outline the kinds of files we envision 
constituting the national data base. These files fall into three categories. 

First, there are wi thin-school micro records. At the minimum, we recommend that these 
micro records constitute a collection of such records in each of a sample of schools. An open issue 
is whether such collections should be censuses in some or all sampled schools. We also recommend 
that such records be contained in at least three linked files: pupil, personnel, and educational activity. 

Second and third, there are school and district-level records. Currently, such records are 
maintained in the Common Core of Data. Under some of the data collection alternatives discussed 
later in this report, these records might consist of samples rather than censuses as at present. Also, 
within district micro records are necessary. At the minimum, samples of personnel records for 
non-school based district personnel are required for a sample of districts so that summary estimates 
of personnel figures do not omit personnel who are not assigned to schools. Financial records may 
also be collected within districts- with coverage of sampled schools-so that human and material 
resource files can be constructed. To keep records of managable size and acceptable accuracy, we 
also recommend that many of the currently collected school- and district-level aggregate counts be 
calculated by aggregating the within-school micro records in the central data base rather than 
continuing the collection of the aggregates themselves at the school- and district-site levels. This 
procedure would allow standardization of data definitions at the micro record level which, in turn, 
will assure the validity of aggregate comparisions. As we discussed earlier, a central problem for 
state-level comparisions currently is the lack of commonality in the definitions of particular data 
elements by districts and states. 

1. Within-school micro records . 

a. Pupil files . These files should be extracted from the various pupil level records 
maintained at the school site. We envision records for pupils with the following 
categories of data elements: 

(i) family background and home environment, 

(ii) special needs and educative difficulties presented to the school, 

(iii) educational outcomes: achievements, graduation, dropout, honors, 
etc. 

(iv) educational participation: attendance, activities, pursuits, 
experiences, e.g., grade level, courses, program participation, etc.; 
these will be linked to the activity files, below. 
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b. Personnel files . These files will include records on the educational personnel 
attached to the school. They would include information on: position held, 
responsibilities, compensation, credentials, academic and empbyment history, 
participation in educational activities mounted by the district. These latter 
elements will be linked to the activity files, below. 

c. Activity file s. These files will include records on the educational activities 
mounted by the school. These activities might be defined at different levels of 
aggregation, e.g., Chapter 1 participation, grade level, semester-class or course, 
counseling services, special activity, p/ogram or course of study. The records in 
this file will be linked to both the personnel records of employees who participate 
in their provision and the pupil records of participants. 

We note here that the number of data elements in each of these files could be quite 
modest We estimate that the physical personnel micro record file required to 
reproduce the current Common Core of Data aggregates need contain no more 
than five data elements each. 

2. School files . These files will contain records for each school in the data base. 
These records will include characteristics of the community served by the school 
as well as organizational and structural charcteristics of the school which are not 
aggregates of pupil, personnel, or activity records and which are not selections of 
district records. Examples of data elements are those currently included in the 
Common Core of Data, however, we envision additional information, such as 
information concerning the community served by the school. 

3. District files . These files will contain records for each district in the data base. 
Data elements will include characteristics of the community served and 
organizational and structural characteristics of the district which are not 
aggregates of within-school micro records or school records. In addition, 
categories, sources, and amounts of in-kind and fiscal resources flowing into the 
district 1 ill be recorded, together with Source-Imposed constraints on their 
expenditure. Expenditures will also be included in three ways: district- wide 
amounts in expenditure categories will be recorded at a finer level of detail than 
currently, micro expenditure data will be collected for district-wide expenditures, 
and micro expenditure data will be collected for sampled schools in the district. 
To supplement these kinds of expenditure data, it is possible to construct parallel 
files on resources puchased. Thus, for example, district level personnel files can 
be constructed for non-school based personnel and facilities, equipment, and 
materials files can also be created. Again, it is important to create these files for 
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resources which are not school based as well as crer 0 school-linked records 
from central district files. Of special importance to tnese latter records are 
eauioment and educational materials. 

A. A. 



Feasible tabulations, analyses, and reports . Two categories of analyses and reports are 
feasible with a data base of this kind. First, there are counts, summarizations and aggregations of 
data elements. Thus, enrollments, achievement test averages, course taking patterns, and home 
environmental characteristics are available by processing individual pupil records to the level of the 
school, the district, the state, and nationally. SinxPar summarizations are possible for educational 
personnel, other purchased resources, educational offerings and resource inflows. At this level of 
analysis, the current information in the Common Core of Data would be reproduced from the new 
data base. However, substantial additional information would also be available which is currently 
unavailable. This includes not only characterizations of pupil background, special needs and 
difficulties, program and course participation and achievement, but also teacher characteristics and 
qualifications, and characterizations of the programs, experiences, and courses offered by the school 
or district. 

Second, the micro data files are linked. This allows relational tabulations and reports to be 
created. In addition to tabulations such as course-taking linked to achievement- a cross-pupil data 
element tabulation referred to above— cross-file tabulations are possible. For example, teacher 
characteristics can be linked- via course taking- to pupil performance. Thus, a three way tabulation 
could be produced from linked micro records of teacher characteristics by course type by 
achievement Such tabulations would form important information for consideration of certification 
or graduation requirements policies. 

D. Access to and Use of the Data Base, 

Conceptually, in order *br the information contained in a data base to be used, it must be 
extracted and transmitted to the person who has use for it. When more than one person has use for 
the information in the data base, the provisions for access can be thought of as a network of 
channels linking potential users to each other and *o the data base. Such a network is complex and 
ought not to be thought of solely as the collection of channels emanating directly from the data base 
itself. 

For example, the wide-spread use of copying machines has resulted in information flows of 
statistical as well as text information to and from potential users who might not have easy or 
inexpensive access under other circumstances. The current rapid expansion of microcomputers and 
telecommunications hardware and software foretells future changes of similar importance in such 



transmission channels. Another example of such flows is through the news media. Important 
information contained in statistical data bases are conveyed via agency press releases and briefings to 
published news reports which reach audiences with no direct contact to the data base. 

Thus, channels of information flew can be thought of as being embedded in a linked network 
wherein users with direct access relay information to other potential users each of whom, in turn, 
may pass it on in edited, selected, or modified form. Each of the individual channels allows 
information flows which have the following characteristics: 

1 . a transmitter and a recipient, 

2. a collection of information to be transmitted, 

3. a format, and 

4. a timing. 

Thus, the issues to be raised in making information available for use relate to: 

1. the group consisting of all transmitters and recipients, 

2. the structure of the communication links among them, i,e., the network, 

3. the way in which the information is modified as it flows through the network, 
and 

4. the matching of the content, format, and timing of the information received to the 
users' information needs. 

Issues in targeting access to information . One of the most important (and most neglected) 
aspects of statistical policy for statistical agencies is priorities for access to information. Part of this 
is lack of information on the part of the agency. Agencies typically do not have a clear conception of 
the actual users of the information they provide, let alone the potential users. Comprehensive 
statistical surveys of the use of the information they provide or the usability of such information to 
relevant populations of potential users are seldom undertaken. Almost all of statistical policy making 
in regard to access has been formed by informal perceptions of existing data constituencies. 

It is time that the Center for Statistics engages in a systematic effort to map out the potential 
collection of users and transmitters, Doth nationally and at the state level. An assessment of the 
information transmission channels would clearly exhibit major blockages in information flow, in 
terms of existence of channels, and in the match of the information content and format to the users 
needs. The Center must create an information base on the actual and potential use of the information 
contained in the national data base and create an ongoing mechanism to set and maintain statistical 
priorities for access. 

Of particular importance at the present time is the timely availability of relevant information for 
public discussion, polLy formulation, and legislative and administrative action at the state level 
With the refocusing of educational policy on state-w z educational retorm, new mechanisms for the 
targeting, presentation, transmission, and retrieval of micro records, aggregate statistics, and 
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analyses based on a federally maintained national data base are mandatory. And how such 
information is channeled to parents, interested groups, and public officials is a fundamentally new 
problem to which federal attention and resources must be allocated 

Existing modes of information transmission and data availability . Statistical agencies have a 
variety of tools for transmitting information to potential users. Our impression is that because 01 the 
lack of information and policy concerning data access, some of these modes may be currently 
underutilized by the Office of Educational Research and Improvement Such access methods may be 
catagorized in the following manner: 

a. Published information 

(i) periodic publication of basic data series: in the 
Center, the main current vehicle for this mode is the 
Digest. 

(ii) serial indicators: periodic reports, topically 
organized, from a variety of sources; the Center's 
vehicle for this is the Condition of Education . 

(iii) topical reports: focused reports on single topics 

of public or policy interest; this mode is not commonly used by the 
Center. 

(iv) pressreleases. 

b. Machine readable files 

(i) public use micro records. 

(ii) machine readable summary files. 

(iii) on-line access to data base. 

c. Response to special data requests 

(i) specially-constructed, user- designed, machine-readable files, e.g., 
micro records, aggregate or summary files. 

(ii) user-requested--pr^:ed or microfiche-tables. 

(iii) user-specified statistical analyses. 

Each of these modes of access is currently being used by at least one federal statistical agency, 
but several are not part of the Department of Education's access and distribution system. The 
following section attempts to link the users of information to potential modes of access 

Goals for a system of information access and distribution . The system of information access 
and distribution is a central policy matter for any government agency maintaining a data base. As we 
discussed in Chapter 4, such a policy involves "who should receive what information, in what form, 
when, and at what cost." Above, we outlined how recipients (i.e., users) of information are linked 
together and to the data base-both by the modes of access supported by the government agency and 
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by th? secondary channels connecting the initial information recipients to others. 

Thus, the tasks of the agency prerequisite to the formation of an information policy are to (a) 
specify and prioritize information recipients and users; (b) determine which information is most 
useful to priority recipients; (c) determine what form such information should take in order to be 
most useful; (d) determine how the utility of this information depends on the timing of its receipt, (e) 
assess how alternative modes of access and transmission determine which users receive what 
information, in which form, when. 

On the basis of these determinations and assessments, the agency can then establish a policy 
for allocating its available resources to the variety of modes of access and transmission in the light of 
their likely consequences for the actual receipt of the information. A major part of this policy would 
be an apportionment of the costs of access between user and agency. Historically, one successful 
effort cf the Census Bureau has been a shifting of portions of the access cost to the user. In the case 
of the Office of Educational Research and Improvement, major users of the information are other 
governmental bodies. This imposes a materially different context on the question of cost 
apportionment than in the case of primary use by the business community. 

The most important issue we raise here is n^l the content of the information policy which 
OERI must establish. It is the necessity of actually establishing one. Until such a policy is 
formulated, there is no basis for designing a distribution system or allocating the agency's limited 
resources to the activities supporting that system. 

E. Data Collection Alternatives 

The proposed information system is based on three elements: 

(1) A cohesive, comprehensive national data base; 

(2) An informrtion access and distribution system, and 

(3) A data-collection system. 
We have proposed, above, 

(1) A framework for defining and organizing the data base, and 

(2) A set of goals and criteria for creating a newly comprehensive system of access and 
information distribution. 

In this section, we identify some alternatives for state participation in the collection of data 
for the new national data system. These alternatives have varying degrees of integration with 
newly-emerging systems of policy and management information at the state level. We also propose 
a new mechanism for cooperative engagement with state departments of education, in forming the 
new national information system. 

We envision data collection as a joint federal-state responsibility. However, every state may 
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net choose to participate in the same fashion. Some states may see little advantage~at least in the 
short run-in greatly modifying the nature arid extent of the management and policy information 
they collect* Others may see considerable benefit in a federally-assisted revamp of their entire 
information system. Most states may be willing to make important changes in the definitions and 
modes of collection of information helpful to them in their policy and management decisions 
without fully integrating their systems with the new national system. 

States choosing complete integration would collect the requisite information from pupils, 
parents, teachers, schools, and local education agencies and forward the data required for the 
national data base to the Federal center. At the other extreme, states may not choose to participate in 
the system at all. For these states, data would be secured via an integrated set of surveys that are: 

(1) federally-conducted, and 

(2) parallel in content, structure and timing. 

Common requirements for data collection. The key characteristic of the proposed data 
collection system is not the manner or mode of state participation. In each state it is the form, 
content, and accuracy of the data collected which is central. Data must conform to the requirements 
of the national data base. In order to obtain-fr each state-such data, some important common 
provisions are necessary. These include: 

1. Identical microrecord d?ta elements will be required for pupils, staff, schools, and local 
education agencies in each state. 

2. The records will be derived from probability samples of all students, staff, schools, 
public and private. 

3. The school samples will provide samples of local education agencies as well as sampling 
frames for samples of staff, students, families and communities. 

4. These sample frames will also provide the source of sub-samples and super-samples for 
conducting other integrated surveys. 

5. Regardless of the source, data, will be collected under sampling and quality control rules 
and procedures promulgated by the Federal center. 

6. Requirements for security and confidentiality will be developed jointly between the center 
and the states. Federal legislation may be requ. ed. 

7. State and local systems will participate with the Federal center to develop a uniform core 
data set to be provided for each state. 

8. Data collection (or capture) will be scheduled to coincide with the important milestones of 
the school year. 

9. The periodicity of the data collection will be determined by the Federal center with the 
cooperation of the state systems. 

A framework for state participation in the system, The level and type cf state participation 
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can follow a large number of alternatives. The choices made by each state will depend on 

(a) the current information system in the state, 

(b) the state's assessment of its emerging information needs, and 

(c) the costs to the state of the participation alternatives. 

The minimum requirement for a national educational information system is that the data from 
each state be comparable. This, in turn, requires that the datadefinitions--i.e., the specifications of 
each data element-be identical from state to state. Thus, if a state is to participate actively in the 
system, it must adopt data element definitions which conform to those specified for the national 
data base. If this is not done, there is no basis for national/state cooperation and the federal Center 
for Statistics must take on the exclusive responsibility for data collection in those states. 

Assuming that common data definitions are established, states have quite distinct existing 
systems upon which to base cooperation. These systems-as discussed above-have 

(a) managerial or regulatory components; typically school district censuses requiring 
transmission of data necessary to monitor conformity to laws and regulations and 
allocate funds. 

(b) policy components; often sample surveys of outcome or process information. 
Because of this, the type and extent of potential cooperation with the national information system 
depends on: 

1. The structure and format of state data collected for policy and management purposes. 
Primarily, this depends on whether the state now collects-or perceives the need to 
collect-micro records on pupils, staff, or activities within schools. Without a current 
focus or perceived need to collect policy or managerial information of this type, state/ 
national integration of data collection cannot be extensive. 

2. The commonality of state and national needs for data of particular types. If there is little 
overlap in the data elements desired for state policy and management decisions~-on the 
one side-and the data elements required for tl*e national data base-on the other-there will 
be little basis for a cooperative endeavor. 

3. The willingness of the state to integrate its own separate data collections and collaborate 
on data collection activities with the Center for Statistics. T o the extent that most or all 
state data collections could be articulated and integrated with the collection of national 
data— i.e., common data forms, tir/es of colle r ion, personnel-a common, cost-effective 
data collection effort (either state or federally managed) would be feasible. 

In order to cbnfy some of the issues relating to mode and extent of national/state cooperation on 
data collection, we havs tried to spell out some of the features at the two opposite extremes of 
national/state cooperation. The first-which we term Alternative A-assumes no cooperation, i.e., 
the Center for Statistics takes on full responsibility for data collection in a particular state. The 
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second extreme-which we term Alternative B-assumes maximal cooperation and integration of the 
state information system with the new national system. We anticipate that most modes of 
cooperation will fall between inese two extremes, at least in thr, initial otages of the new system. 

Alternative A: federally-conducted sample surveys. Under all of the alternatives, the data 
records wili be produced from an integrated set of parallel, consistently timed, state representative 
sample data collections of individuals and administrative records. Under Alternative A, this will 
take the form of sample surveys conducted by th~ Federal Education Data Center. In each state the 
C ..:r would select a p A lability sample of schools that will provide the basis for any other 
samples which may be required. This could include samples of students and staff, families of 
students, and school districts. In addition to public schools, a probability sample of private schools 
would be included in the system. Therefore Alternative A provides for the content, sample design 
and data base organization of the new education data system. This section describes the critical data 
collection elements which describe the fully Federally managed system. 

The sample . As noted above the samples for the new system will be state representative. 
The samples will, however, conform to probability design so that it will be possible to examine 
various organizational constructs and geographic units other than states. A : or example, the resulting 
data file could be subdi/ided to focus on urban schools in a particular state or group of states; 
predominantly minority schoc. 7 may be examined; students or teachers with certain characteristics 
could be examined separately. The principle limiting factor would be the size of the subsample 
under study. 

The sample design for the surveys Should provide for sample rotation so that a particular 
schc * 1 might be in the sample for only a few years. Each year a subset of the schools could rotate 
out of the sample and be replaced with another. Although some large or special schools may be 
permanently included in the sample with certainty, the number of certainty schools may be kept to a 
minimum. However, this rotation scheme could introduce some disturbances into the data. Biases 
might result when schools remain in the sample for more ti;an one series of interviews. The Center 
should anf ; -:ipate this possibility and develop a research program which would examine the data 
from each subset separately. 

Questionnaires and other instruments . Under this alternative, data w! 1 be supplied directly to 
the Fe^ fcr al Data Onter or its agent from the survey site. Various methods \ /ruld be used to collect 
data. N * >enever the subject matte? permits, direct face to face interviews would be conduvteu. 
Other meihocb such as mail o~ telephone ' iterviews may be used as appropriate. 

The C iter will develop the methods to be used for each survey and will design the collection 
and control instruments accordingly This will require an exteroiw research and development 
effort along with a full program of testing. Every effort to determine the v. liciity of responses to 
the collection instruments should be undertaken at this time. Response burden, cost, and timeliness 
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must aJso be considered when the instruments are being designed to find the least burdensome and 
costly way to produce valid and timely data. 

Each document will be the basis of a microrecord in the final system and as such should 
allow for the kind of flexibility needed to assign survey units to categories according to the Center's 
definitions. For example, according to The Center for Statistics' staff, at least one state does not 
classify an excused absence as a true absence; with the collection of summary data, this would 
present either a comparability problem or a problem of dual record keeping. Using a microrecord 
approach, the questions for a particular pupil, on the sample day, in the sample school, should ask 
whether the student was physically present (de facto) and if not, whether the absence was 
"excused" (de jure) . The pan of the pupil questionnaire addressing these issues might appear as 
follows: 

Was ... (pupil) ... physically present in school today? 
[ ] Yes — (go to next question) 
[]No-- (ask:) 

Was this an excused absence? 

[]Yes 

(Go to next question) 

[]No 

Aggregating these two pieces of information for each pupil in the school or state education agency, 
the Center may sort out this relatively simple definitional problem. In addition ii might be possible 
to develop useful information on truancy. 

The data for each pupil would also include the basic characteristics of the pupil - age, sex, 
race, course information, grade average, test scores and other information which will provide 
demographic data, performance information and other information about the pupil which when 
aggregated will provide extensive information about all pupils. Further, the pupil information will 
be related to that of the pupil's teachers, his or her school, and the loc^i education agency to 
provide a complete picture < ; the educational system. Because the development of the forms, 
documents, and linkages is not simple, however, an extensive research and testing program will be 
needed. The designer of the pupil microrecord will have perhaps dozens of such issues to 
confront. 

Similar issues will have to be considered for the microrecords for staff, schools, 
communities, local education agencies, state education agencies, and any other unit of sample to be 
included in the new system. The Center will require a unit to develop the micro record forms, 
questionnaires, or schedules. That unit will have to work closely with the states (and to the extent 
necessary with private schools or school groups) to determine th * detailed questions needed, and to 
examine definitional differences between different state and local systems. Although these tasks are 
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not easy, there are many precedents within the Federal government; most of the Census Bureau's 
work involves the development of questionnaires to produce microrecords. Similary the Bureau of 
Justice Statistics, the National Center for Health Statistics and others have extensive experience in 
dealing with these issues. In the final analysis, >:andard questionnaires to produce microrecords in 
the hands of highly trained interviewers should substantially reduce problems of non-comparability 
and improve data quality in general. 

Quality Control . "Garbage in garbage out" has become a cliche in the computer age; 
nevertheless its truth has never been disputed. It is essential that the Center develop procedures to 
prevent "garbage in" ?r each state of the process. If the new system is to be effective, quality 
control procedures must be introduced at the time the survey instruments are being developed. 
These should be rigorous pilot studies and pretests of all of the instruments and all of the 
procedures to be employed in the field collection. 
< "Hot houses" should start the process. This is a simple process in which convenient 

possible survey subjects are asked to respond to potential survey questions. Pilot studies are more 
formal and typically involve a purposive sample of potential survey respondents. The pilot should 
uncover more subtle types of problems with the instruments. The pretest should be designed to 
begin to identify measurable statistical problems with the instruments, the procedures and even the 
sample; validity question ^ .ould begin to be a. 1 dressed at this stage. Frequently, there must be 
repeats of any or ail of these pre-enumeration activities. Even after all of this testing has been 
performed and analyzed and the instruments have been put in . j use, problems with the 
questionnaires and procedures will become evident. A constant monitoring program must be 
established to determine whether the instruments had an initial defect, whether there has been a 
measurable biasing effect of pre-enumeration or whether the "world" has changed. 

Alternative A us>»*; interviewers to collect data dir- ciiy; there must oe the establishment of 
qualification standards for new interviewers involving appropriate test* or other selection devices 
along with a system of initial supervisory field observations. After new interviewers have become 
qualified they wil! still continue to be observed on a systematic basis to ensure that the survey's 
procedures are being followed. The Center should also develop a system of regular re-interviews 
in which supervisory personnel will re-interview a small systematic sample of all completed work. 
Acceptable error levels will have to be determined. Any interviewer whose work fails re-interview 
may be retrained, terminated or dealt with in other ways as appropriate. The re-int^rview sample 
will also provide a basis for estimating non-sampling error. 

Since data will be received directly by the Center or its agent, the Center will be responsible 
for all data processing, including coding, data entry, editing, weighting, and developing the data 
base. All of these procedures must also be subject to rigorous quality control. Samples of -:cded 
materials will be subsampled arid the coding verified. Similar steps wiil be taken for data which are 
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keyed or otherwise encoded. 

Organizational issues . As noted above, the responsibility for data collection lies with the 
Center. It can opt to undertake the collection directly by acquiring field staff and data processing 
capability. However, the cost and development times sharply limit the practicality of this option, 
particularly in the short term. Several commercial contractors already possess the technical and 
logistical capacity to carry out this work, however Federal procurement policies make it difficult to 
develop the long-term relationships which would be desirable for program development and data 
consistency over time. An option would be to ask another Federal agency to undertake At survey 
series under a reimbursible agreement. Under such an agreement, the Center could build a 
long-term relationship which would permit the development of integrated surveys which would 
respond to the changing needs for education statistics. Although Hill states that "contractors can 
usually get better respondent cooperation than federal agencies can/' the reverse has been true of 
4 those Federal agencies which are clearly identified in the public mind as statistical collection 
agencies. This is particularly true of the Bureau of the Census which continuously has response 
rates for its reimbursible surveys as high as 95%. 

Even though it is likely that the Center will contract out the data processing, it should have a 
staff capable of preparing specifications fcr data processing, weighting, and any other technical 
procedures. 

Respondent Cooperation . In order to obtain respondent cooperation and to keep the 
aggrevation level at a minimum, the Center will have to take several positive steps. For example, it 
should carefully communicate its rotation policy to the schools so that they will understand the 
relatively short term nature of the commitment. Moreover, the Center will have to develop a 
program through which at least the sample schools receive pre^imin ry reports of the data collected 
for tlivir statistical area so that they will be able to make comparisons with similar schools. It 
would be useful for the Center to develop a package which would show individual schools how to 
utilize data in their own planning and development efforts. Perhaps a subcommittee of the 
consortium discussed below could assist in the development of such a document. The Center must 
also be responsible for keeping respondents apprised of future survey visits with information 
about the content and the timing of each inquiry. 

Alternative B: Integrated State Policy and Management Information Systems . Several states 
arc in the process of developing integrated state-wide policy and management information systems. 
Some of these states may be interested in providing data to the Center through these systems. This 
would be particularly desirable if certain conditions are met. First a state would have to agree to 
provide an identical set of data elements for each school in the sample and the lojal education 
agency associated with each school. The federal data base requires an integrated set of micro 
records on a statf epresentative sample of schools. In designing a state pclicy and management 
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information system, to be integrated with the national system, states may wish to explore options 
with respect to (1) census versus sample survey modes of data collection and (2) different levels of 
integration with their own systems. In any case, the states would have to agree to provide the data 
on a schedule which is consistent with the processing and publication schedule of the Center. 

In the discussion above — Alternative A — we treated issues related to the design of 
questionnaires and other instruments and to the quality control of the data collection activities. 
These issues are also central to a data collection system fully integrated with the state management 
and policy information needs. The same thorough pilot testing and control modes are required. 
However, these must be developed jointly by the Federal center and the state. 

Other Alternatives. Because the proposed system is to some extent modular, there could be 
a variety of possible state participation modes. At one extreme a stat^ could provide all of the data 
required for the entire system, including test and other survey information, to the other extreme in 
which a state would not actively participate at all. Some states would have a census of all of the 
administrative record information in the system, while others would only maintain the data for the 
Federally defined sample, depending largely on whether the system is to be used for management 
information or policy information. 

There would be no restrictions on any state's system as to supplementary data elements and 
sub- systems it may wish to develop. A state would, however, have to provide to the Center only 
data for the schools in a sample and only those data elements requested by the Center. While it 
might be easier for some states to provide a computer "dump" of all of the data in its system, it 
would not be appropriate for the Federal Center to hold all such data in its computer. 

Interested states would ask to supply machine readable data to the Center. It would be the 
responsibility of the Center to determine when a state system had reached the point where ts data 
were complete, consistent, and accurate enough to enter the system directly. 

The quality control requirements of direct state data entry into the Centers statistical 
program would have to be rigorous. Annual audits of a random sample of schools would be part 
of the program; some individual schools also be included with certainty if there was any indication 
of previous problems with data from that school. The Center would have to retail the authority to 
modify or discontinue a state's direct participation if the quality control procedures indicated serious 
data problems. 

The data currently collected in NELS, NAEP, and other surveys would also be integrated 
into the new national data system. 

A Mechanism for Cooperative Engagement . In order to develop the detailed design for the 
new national data system, the Center, working through the Chief State Scnool Officers, should 
establish a consortium of all states and develop an agenda foi* identifying specific information 
elements and data elements required for the system. The Center should also appoint a number of 
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other members to the consortium, including representatives of local education agencies, academia, 
and other users concerned with information about the educational system. The consortium should 
also have a say about the method in which the data base is organized and what data, in wLat form 
would become available. 

It would be foolish to believe that a body representing this large a constituency could do the 
derailed planning required for this effort. There would be an obvious r ed to develop working 
groups to address specific issues. For example, several states are in the process of developing state 
level integrated information systems; each is designed to provide the specific data needed for state 
purposes. In order to foster the development of compatible systems, the Center should attempt to 
organize a working group of the consortium consisting of states already developing such systems, 
along with other states interested in similar development. Since in general the systems would be 
integrated, it would be essential for local systems to be represented. This would facilitate the 
exchange of information among the states and the development of alternadve models which could 
feed the national data base. 

Although the Center would have the responsibility for staffing the co" ^ortium and 
establishing working groups for the various technical issues which will have to be addressed 
during the development of the system, the total input to the Center should more than compensate 
for the cost of staffing. 



F. Relative Costs and Benefits of Alternative Designs 



Types of data system costs . Data systems are costly and beneficial in several common ways, 
and in this regard, differ principally in the relative amounts, rather than the types, of their costs and 
benefits. Development of a new education data system will be costly, in part to the federal 
government, in part to state governments, and in part to local education agencies. Alternative 
education data systems differ not only in the absolute magnitude of their development costs, but in 
the distribution of these costs across levels of government. The same can be said of the costs of 
maintaining an education data system, once established. In this section, we describe .he types of 
costs that must be borne in establishing and maintaining an education data system, and the ways 
these costs impinge on all levels of gove/nment. As will be seen, not all types of costs evidence 
themselves directly in dollar amounts. Some involve investments of personnel, others impose a 
burden on data providers, and yet others must be counted in terms of foregone opportunities. 

Development costs to the federal government . In any national data system on elementary and 
secondary education, it is anticipated that a large portion o r the dollar outlay costs of developing 
the system will be borne by the U S. Department of Education. The types of development costs 
incurred will include those associated with specifying the content of the system, selecting modes of 
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data collection, identifying potential providers of information, conducting needed research on 
methous of measuring certain variables and methods for securing certain types of information, 
developing and validating instruments for data collection, developing plans, mechanisms and tools 
for analyzing data and for reporting results, and developing procedures for data collection, 
processing, analysis, and reporting. 

To state governments. To the extent that they choose to integrate a newly-developed national 
data system on elementary and secondary education with their current management information 
systems and other data collection activities, a portion of the costs of developing a new system 
would be borne by state governments. It is likely that state education agencies would incur dollar 
outlay costs in modifying existing state data-collection instruments, in modifying data transmission 
channels and procedures, in training providers of data to participate in the new system, in 
developing plans, mechanisms and tools for analyzing data for state purposes and for reporting 
results for various state purposes, and for developing administrative procedures for integrating strte 
data systems with that of the nation, in terms of data collection, analysis, and reporting. 

To local education agencies. Whether local education agencies would bear a portion of the 
dollar outlay costs of developing a new national education data system would depend on their 
states 1 decisions regarding integration of state management information systems with the national 
data system. If states choose to integrate their systems, local education agencies would be require^ 
to modify the procedures by which they collect, analyze and store information for the purpose of 
reporting to their state education agency. Such changes would carry certain collar ouriay costs. In 
addition, local education agencies might also choose to modify their management information 
systems so as to minimize redundancy with, ana make best use of, a rational d?*^ system on 
elementary and secondary education. With decisions of this type, local educatnn agencies would 
incur the same types of dollar outlay costs enumerated above for state education agencies. 

Maintenance costs and personnel requirements to the federal government. The costs of 
maintaining a national data system on elementary and secondary education would, regardless of 
system design, be shared among the federr 1 government, state governments and local education 
agencies. However, the distribution of cos^s among these levels of government would likely vary 
subscantially, depending on the functions to be served by the data system and its consequent 
design. Maintenance costs that are likely to be borne by the federal government under any design 
almost certainh would include the cost of collecting, processing, analyzing, and reporting 
educational data for purposes of informing federal education policies. The federal government 
would also bear the cost of maintaining and operating the national data base. It is reasonable to 
expect that, at a minimum, nationally-representative samples of respondents would be required to 
satisfy such purposes. Another type of maintenance cost likely to be boine by the federal 
government under all system designs would be the cost of ongoing research and development 
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needed to improve the validity and precision of data collected through the system. 

Even if most of the data collected through a national data system on elementary and 
secondary education were secured through contracted sendees, the U. S. Department of Education 
would have to maintain an in-house staff with sole responsibility for management of the system. 
Whether an integrated data system would increase federal staff requirements or reduce them, 
compared to the staff needed to maintain current NCES data-collection projects, is difficult to 
anticipate. Nonetheless, investment of federal funds alone would be insufficient to develop and 
maintain a new education data system. Since the utility of an integrated data system, as proposed 
here, would far exceed that of the entire collection of current NCES projects, it is reasonable to 
anticipate the need for additional personnel to satisfy the information requests of large numbers of 
presently unserved users. 

To state eovernments. Regardless of data system design, state governments would likely 
• incur some costs in supplying data needed to maintain a national education data system, ^o the 
extent that states chose to integrate their education management information systems with a national 
data system, or to expand the coverage of a national education data system so as to meet their own 
needs for education policy information, these costs would increase. In either of the latter cases, 
state governments would incur ongoing costs for validation of data quality, data analysis, and 
reporting of results. In addition, states would incur costs for training suppliers of data. 

The state personnel requirements associated with a national education data system would 
depend almost entirely on a state's chosen level of participation in the system. If a state were 
content to assist the U. S. Department of Education in securing information required for federal 
policy purposes, the state personnel requirements of the data system would be minimal. However, 
if a state chose to expand the national data system so as to meet its own needs for education 
management information and/or policy information, the need for state personnel with responsibility 
for development, maintenance, ari management of the integrated data system would likely be 
substantial. 

To local education agencies. Since local education agencies would be among the principal 
suppliers of cata under any design for a national education data system, they would certainly incur 
costs associated with collecting, storing, and reporting data. If local education agencies choose to 
modify their management information systems so as to minimize redundancy with, and make best 
use of, a national data system on elementary and secondary education, they would incur additional 
maintenance costs for collection of data, validation of data, analysis of data, and reporting of 
results. 

Local education agencies are unlikely to incur substantial personnel costs as a result of their 
participation in a national education data system that is designed solely to meet federal requirements 
for education policy information. However, participation in an integrated system designed to meet 
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state needs for management information as well as federal needs for policy information might 
require significant local agency investment in personnel with data systems experience. This would 
**!so be true if the local education agency decided to merge its o w n management information system 
with the national data system. Collection of data for management purposes implies complete 
coverage of the units to be managed, rather than sampling of units. An integrated education 
management system that was based on micro records of the sort proposed here for a national data 
system would very likely secure comparable data on all students enrolled within the local education 
agency. In a large local education agencies, requirements for personnel to maintain such system 
would be substantial, but might not exceed the requirements of current, fragmented NCES, local, 
and state data-collection projects. 

Data access costs at the state level It is reasonable to assume that state governments would 
incur some costs for accessing information from a national education data system that was 
developed and maintained by the U. S. Department of Education. The magnitude of these costs 
would likely vary substantially, depf.* ling on the state's contribution to the development and 
maintenance of the data system and the nature and form of the state's needs for information. 

At the local level. If a local education agency wished to secure information from a national 
education data system, it too would likely incur some costs, depenuiag on the nature and form of 
the information required, and the agency's contribution to the development and maintenance of the 
data system. The costs of access to publicly available reports and standard data tapes would, in all 
likelihood, be trivial. But the cost of access to data that required special analyses or the construction 
of special, non-standard data sets, might be considerable. 

Respondent burden for local education agencies and schools. One type of non-dollar cost 
that varies only by degree across designs for a national education data system is the burden borne 
by respondents to requests for information. Providing information is time-consuming, and 
therefore impinges on other, and sometimes more fundamental, activities of the schools. Accurate 
assessment of the respondent burden imposed by a national education data system will be difficult, 
since a complex system that is designed to serve a multiplicity of purposes is likely to impose the 
greatest burden, while reducing the total bu:den on respondents that would be imposed by the 
separate, unarticulated, requests for information that presently originate at alMevels of government. 

For state governments . In addition to local education agencies and schools, state 
governments (most likely state education agencies) would, regardless of data system 
design, face some burden of providing data for a national education data system. Certainly, the 
burden attributable to the data system would increase if a state ch )se to meet its own needs for 
education policy information or management information by expanding the national data system. 
However, as noted above, the efficiencies realized through data system integration would likely 
reduce the state's total expense and time investment in the collection, processing, and ana ? /sis of 
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data, and reporting of information on education. 

Opportunity costs and associated response burden. Every data-collection activity carries 
opportunity costs of several kinds. Funds invested in the data system are not available for use in 
other projects. Personnel re quired to develop, maintain, and manage the data system are prevented 
from pursuing other goals. The good wil 1 of potential respondents is expended on the burden 
imposed by the data system, and alternative requests must be minimized. 

Types of data system benefits. The benefits t be realized from a data system can be 
characterized in many ways. Most fundamental are the benefits inherent in the ready availability of 
pertinent, high quality data. Another route to characterization of benefits is through consideration of 
the uses to be made of the data supplied. For example, benefits accrue through the use of data to 
increase the effectiveness of policy aralysis and policy formation, or to increase the effectiveness 
and/or efficiency of management 

The relative costs and personnel requirements of alternatives A and B . Alternatives A and B, 
discujsed in Section 5F, ue extreme options for state participation in the national data system. 
Here we characterize these alternatives with respect to costs and, in the section following, the 
benefits accruing from them. 

As described above, Alternative A consists of an integrated survey of state education 
agencies, local education agencies, schools, teachers, students and households that will produce 
micro records containing linked files of information on the topics specified in Chapter 4 of this 
report The survey would be developed and maintained by the U. S. Department of Education, 
through one or more contracts with professional survey research organizations. The survey would 
produce basic education statistics and education policy information required by the federal 
government to meet its Congressionally-mandated responsibilities to collect, analyze, and report on 
the status and condition of education in the United States. Ultimately, the survey would provide 
comparable regional and state-by-state education statistics. 

Alternative B consists of a combination of data-collection activities that would be put into 
place in successive phases. The integrated data set described unc'er Alternative A would be at the 
heart of Alternative B as well. In addition, through a coop *raiive project with the U. S. Department 
of Education, a consortium of states would develop a common data program for the principal 
purpose of collecting information needed to manage thiir education systems. Ihese management 
information systems would be linked to the national data system by collecting and scoring micro 
records on variables that complemented inose collected through the national data system and, 
eventually, when all 50 states adopted a common management information system, state and federal 
needs for policy information could be accommodated through the state systems and the need for 
federally-operated data acquisition projects would be eliminated. 

The primary costs: of developing the data system proposed under Alternative A would be 



86 

92 



borne by the federal government. Although some limited information would be sought directly 
from state education agencies, most of the data collected under Alternative A would be collected 
from local education agencies or from schools and individuals within local education agencies. 
Therefore, state education agencies would not be required to alter their current data collect! n 
activities under Alternative A (although it would be to their distinct benefit to do so), and 
consequent development costs at the state level would be minimal. In contrast, a substantial portion 
of the costs of developing Alternative B would be borne by state governments, since current 
management information systems or projects designed to secure data for the purposes of education 
management would have to be modified. Even if the federal government provided major funding 
for the development of new data collection mechanisms and instruments under Alternative B, the 
development costs to state agencies would be far larger than those imposed by Alternative A. 
Systems for analysis of data for state management purposes would have to be developed or 
modified, as would systems for data verification, storage, management, and retrieval. 

Both Alternative A and Alternative B would impose some development costs on local 
education agencies and the schools within them, since, in many cases, definitions of variables in a 
national data system would difter from those previously used by th. .gencies and schools. 
Changes in definitions would necessitate some changes in the data collected by schools and local 
education agencies, as well as changes in the ways some records are aggregated and stored. 
However, a national data system that was designed principally or exclusively to provide basic 
education statistics for the states and the nation, in addition to information for federal policy-making 
purposes (Alternative A), would impose lower development costs for local education agencies than 
would an alternative that included collection of data for state and/or local management of education 
(Alternative B). 

The primary costs of maintaining the data system proposed under Alternative A also would 
be borne by the federal government. Since the required involvement of state education agencies 
would be minimal, their required dollar contribution to the maintenance of the data system would be 
minimal as well. In contrast, a major portion of the costs of maintaining the data system proposed 
under Alternative B would also be boine by state education agencies, particularly as an increasing 
number of state agencies adopted components of the system that were designed to provide 
information for management of their educational systems. 

As was true of development costs, some cost of maintaining the data systems proposed 
under both Alternative A and Alternative B would be borne by local education agencies and the 
schools within them. Micro records that were consistent with the definitions imposed by the data 
systems would have to be compiled and maintained in these administrative units for many 
categories of variables. Again, for a data system that provided basic statistics on education at the 
national level, in addition to satisfying relatively limited federal policy purposes (Alternative A), 
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these costs would be lower than those associated with a data system that served the multiple 
purposes of federal education policy analysis, state education policy analysis, state management of 
education and local management of education (Alternative B). 

It is reasonable to assume that the cost to local education agencies for accessing the national 
data system on elementary and secondary education would be virtually identi under Alternatives 
A and B. In either case, it is likely that the federal government would, at most, attempt to recover 
its expenses in meeting requests for information on education. 

Clea^y. Alternative B would require a far larger investment in data system personnel by state 
education agencies than would Alternative A. Alternative B would require an extensive staff in 
each state education agency with responsibility for developing and maintaining that state's 
education management information system, and for integrating that system with the national data 
system on elementary and secondary education. In some states, state agency personnel who are 
currently responsible for educational data systems might be able to develop, integrate, and maintain 
a new system, obviating the need to hire and train additional personnel. 

Alternative B would also impose somewhat greater personnel costs for local education 
agencies and schools than should Alternative A, since, as noted above, an education management 
information system would require collection of data from all schools in a state, and possibly from 
or about all teachers and all students in a state. In contrast, data sufficient for the dual purposes of 
compiling basic statistics on the status and condition of education, and informing federal education 
policy, could be collected from representative samples of these populations, and would not require 
complete coverage of the local education agencies in the United States. The increased volume of 
data and the greater range of data to be collected under .Mternative B would impose additional 
personnel requests in local education agencies that supplied these data. 

Alternative B would also impose larger opportunity costs on state governments and local 
education agencies than would Alternative A. Since substantially more data would be collected 
from and by state education agencies and local education agencies under Alternative B (compared to 
the requirements of Alternative A), this option would preclude a greater range of alternative data- 
collection activities. In addition, a consequence of the larger capital investments required of state 
governments and local education agencies by Alternative B is a greater range of foregone 
opportunities to collect and analyze data on education by other means. 

The relative benefits of Alternatives A and B . Since an integrated national survey is central to 
both of the data system alternatives described above, the alternatives are virr jally indistinguishable 
on severed of the benefit dimensions discussed earlier. Once fully developed and installed, the 
alternatives sys^ns would be equally beneficial in terms of (1) timeliness of data - the lag between 
data collection and reporting could be comparable under both alternatives, since, for purposes of 
reporting basic statistics and analysis of federal education policies, identical data would be 

88 



9i 



collected; (2) validity of data in the survey core, identical data would be collected under both 
alternatives, leading to comparability on this dimension; (3) intestability of data common 
codin^ systems that would ensure the integratability of data collected on local education agencies, 
schools, teachers, students, and households could be used under both alternatives; and (4) 
accessibility of data - presumably, identical information would emerge from the sample survey that 
is common to both alternatives, and identical ranges of options for accessing data collected through 
the system could be provided under both alternatives. 

A central feature of a fully developed and installed Alternative B is its complete coverage of 
locai education agencies and schools within all participating states. With 50-state participation, the 
system would provide the potential of complete coverage of local education agencies and schools 
throughout the nation, on variables that were essential to state education management objectives. 
Since data on a greater variety of variables would be collected under Alternative B, as well as data 
on a far larger number of administrative units, Alternative B would exceed Alternative A on many 
of the benefit dimensions discussed earlier. These include: (1) availability of data — data would be 
collected for a larger number of variables; (2) precision of data - either complete coverage or larger 
samples of units would be used under Alternative B, leading to increased estimation precision; (3) 
accuracy of data — complete coverage of administrative units would eliminate inaccuracy due to 
inadequate sampling, and might help control other sources of inaccuracy in data; (4) 
disaggregability of data — again, complete coverage of administrative uirts under Alternative B 
would ensure that data could be disaggregated to virtually any desired level; (5) representativeness 
of data — complete coverage of schools and local education agencies would ensure the 
representativeness of data pertaining to these types of units; (6) aggregability of data - with 
complete coverage of schools and local education agencies, Alternative B would ensure the 
aggregability of data to all higher- level administrative units and to all subpopulations for which 
identifying information had been collected. Since data would be collected from a sample of 
administrative units under Alternative A, the aggregability of data to subpopulations of interest is 
not certain; (7) utility of data - by design, Alternative B would secure data for a larger variety of 
purposes than would Alternative A. The enhanced utility of data collected under Alternative B 
would thus be ensured. 

G, Development and Phasing o f the Data System 

Transcendent development and phasing issues. Regardless of the levels of involvement of 
individual states, development of a new national data system on elemental / and secondary 
education will substantially affect current data collection, analysis and reporting activities in state 
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education agencies, in local education agencies and in the U.S. Department of Education. As noted 
earlier, a new data system will require dollar resources and personnel currently allocated to ongoing 
federal projects and will impose respondent burdens mat will preclude the continuation of 
numerous existing federal data-collection projects and the initiation of others. Issues such as these 
transcend the selection of a data system design, and influence the phasing and timing of data system 
development and installation. The most pertinent of these issues are: 

1. Preservation of essential data time series 

2. Requirements for research needed to develop critical elements of the new data system 

3. Provision of adequate time for data system testing and verification 

4. Impact on current state and local education agency data systems 

5. Cost and personnel requirements of various phases, by level of government 
Preservation of essential data time series. Although this report has identified many 

inadequacies in the present national data system on elementary and secondary education, the 
Federal Center for Statistics has, nevenheless, maintained several essential data time series. 
Examples of such time series car be found in the Center's Publications, The Condition of 
Education and the Digest of Educational Statistics, and include, by way of illustration, total 
enrollments at all levels of education throughout the United States which have been reported since 
1899-1900 (Digest of Educational Statistics. 1983-84. Table 3, p. 8). Such time series must be 
preserved in the new national data system. 

Requirements for research needed to develop critical elements of the new data system. Most 
elements of the national data system we have proposed can be developed using existing survey and 
measurement technology. However, other elements will test the current state of the survey and 
measurement art and will require intensive research and development. For example, our proposed 
system requires micro record information on a variety of educational outcomes, including but not 
limited to, achievement test data. To secure such data while adequately controlling the respondent 
burden imposed on individual students will require the development of new, highly efficient 
outcome measures anu new approaches to the use of matrix sampling. As a second example, our 
proposed system calls for dmely production of policy-relevant analyses that are responsive to 
immediate and particular requests from the broad array of information users, including policy 
makers in all branches and at various levels of government, as well as the new constituencies 
described in detail in Chapter 1. Considerable research is needed to develop mechanisms that will 
enable the Federal Center to meet these immediate and particular requests for information with 
timely, valid, and responsive policy-relevant analyses. Research is needed in such areas as 
verification of the requestors' rights to data access, mechanisms for rapidly and accurately building 
relational analyses using data stored on the basis of distinct file structures, methods for providing 
users with a variety of alternative relational analyses, and methods for assessing the relative utility 
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to policy makers of such alternative analyses. 

Provision of adequate tim e for date system te sting and veri ficajjorL Because many features 
of the data system we have proposed are novel, extensive testing and verification of mechanisms 
for data collection, data aggregation and storage, and information retrieval will be required. Such 
testing will require a continuing commitment by the Federal Center for Statistics, and. the continuing 
cooperation of state and local education agencies over a period of years. Such agencies must 
acknowledge and agree that no element of the data system will be used operationally until it has 
been throughly tested and its quality has been fully verified, regardless of the sense of urgency that 
pervades present efforts to provide information on the nation's education systems. 

Impact on current state and local education data systems. The burden imposed by the new 
data system on state education agencies will vary depending on an individual state's level of 
participation. At one extreme, if a state chose to keep its own data systems completely separate 
from the national data system, no additional data burden would be imposed, nor would the state be 
required to adapt its data-element definitions to be coincident with those of the national system. A * 
the other extreme, if a state chose to fully integrate its data systems with the national system, it 
would have to accept and adopt the data-element definitions used in the national system, and it 
would have to adopt the micro record structure that is central to the naional data system. In 
addition, in some of the smaller states, the proposed national data system will likely collect data in a 
substantial proportion of the states 1 local education agencies. Therefore, th^ response burden 
imposed by the national data system might limit the data-collection options of the state education 
agencies in these states. 

The impact of the national data system will be limited to those local education agencies that 
are included in the state-representative samples used by the system. In sampled local education 
agencies, the volume and density of data collection envisioned for the naitonal system will represent 
substantial data burden and will likely require local agencies to adapt their own data systems in 
several ways. For example, the local education agencies might choose to make the data element 
definitions they use consistent with those of the national system. Local agencies might also choose 
to make the structure of their data systems consistent with the micro record structure of the national 
system. Although such modifications v/ould not be mandatory, they would help to rmnimizt 
respondent burden and to maximize data collection efficiency. And, of course, in states that chose 
to integrate their data systems with the national system, these impacts would not be limited to a 
sample of local education agencies, but would apply to all of the states' local agencies. 

Cost and and personnel requirements of various phases, by level of government, In any 
national data system on elementary and secondary education, a large portion of the dollar outlay and 
personnel costs of developing and maintaining the system would be borne by the Federal Center for 
Statistics. This would be true at all phases of development. In particular, the costs of necessary 
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research and development at the national level would be borne entirely by the federal government, 
as would the costs of testing and verification. 

As noted above, the state personnel requirements associated with a national education data 
system would depend almost entirely on a state's chosen level of participation in the system. And, 
to the extent that a state chose to integrate its data systems with the national data system, it would 
share in the cost of developing the national data system. Such cost sharing should be balanced with 
the relative benefits accruing to the parties involved. 

Dollar outlay costs and personnel costs incurred by local education agencies would depend 
on the degree to which they chose to integrate their own data systems with the national system and, 
to some extent, on the degree to which their state chose to integrate its data systems with the 
national system. If neither the state agency nor the local agency chose to integrate their data 
systems with the national system, the dollar outlay and personnel costs incurred by a sampled local 
education agency would be negligible during the devleopment of the national system, and would 
not te substantial once the system became operational. In a state where the state agency chose to 
integrate its data systems with the national system, data for state management purposes would be 
collected from every school and local education agency in the state, and local education agencies 
would have to bear any consequent increased costs to meet the state's data needs. Efforts should 
be made to minimize-on a continuing basis such cost increases. 

Development and phasing of the federal component 

Phases of development. We propose that the federal component of the national data system 
be developed in distinct phases encompasing a five-year period. Phases would be distinguished by 
specified calendar periods. Within each phase, the specific categories and subcategories of data 
elements that compose the national data system would be at different levels of development. Data 
elements would differ in terms of their availability for operational use and the level of their 
aggregation. Some data elements would be objects of research and development; other data 
elements would have advanced to a field testing and verification stage; still other data elements 
would have been tested and verified in earlier phases, and would be available for operational use; a 
fourth category of data elements would not yet be available in any form. In addition, some data 
elements would be available initially only from aggregate records, while others would be available 
in the form of micro records. The phases would also be distinguished by the numbers and types of 
data elements that could be linked across data categories and data files, as micro records become 
available for operational use. 

Categories of data. The categories and subcategories of data that will compose the national 
data system will be drawn directly from the conceptual model described in Chatper 4 and elaborated 
in an earlier section of this chapter. The major categories include: environment (commuruy and 
family characteristics and expectations); incoming resources (financial revenues and other incoming 
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resources for schooling); educative difficulties (pupils' capabilities, motivations, handicaps, 
English language facility, out-of-school supports, etc.); educative goals (school goals and 
objectives, curriculum ); allocated resources (facilities, staff, equipment, materials, and other 
allocated and purchased resources); educational pursuits (curricular offerings, standards, 
teaching-and school-related activities); participation (pupil participation in the process of schooling); 
and outcomes (achievement, graduation or dropping out, political participation, employment). 

Calendar periods . We propose that the first phase of development, following the 
establishment of the consortium described below, begin on July 1, 1986 and extend to December 
31, 1986. The second phase of development would begin on January 1, 1987 and extend to June 
30, 1987. Subsequent phases would encompass six-month periods thereafter, through June 30, 
1991. 

Status of data elements. In any phase, each data element that will ultimately be a part of the 
* national data system can be characterized as belonging to one of three categories of development. 
At one extreme, would be data elements that are not yet included a any form. An intermediate 
category would be data elements that were collected only in aggregate form; e.g., school 
membership determined from a report prepared by a school. Note that we do not propose to 
develop such aggregate reports, only to maintain specific elements that are ci Tently a part of critical 
NCES data-collection activities, until they could be replaced by tested and verified micro records. 
These micro records would constitute the third category (that is, the other extreme of development) 
of data elements. Currently, such micro records only exist in data projects such as NAEP or 
NELS. 

Availability of data elements. We believe that most, if not all, of the data collection formats 
for elements required in early phases of implementation of the system already exist within current 
Center data programs (e.g., NAEP or NELS). Development of data collection formats and 
activities incorporating such data elements formats would be required to implement the system, but 
more fundamental "research" activities would not At any phase of development, however, a data 
element that existed in micro record form misht not yet be available for operation use. Initially, 
some micro record data elements would require extensive research and development. Once a data 
element has been newly devleoped, it would be subject to extensive field testing and verification; 
that is, a new data element in micro record form would not become a part of the operational data 
system until convincing evidence of its validity and utility has been amassed. Only after the 
validity, utility, and feasibility of collecting a data element in micro record form had been 
demonstrated, would that element be available as an operational part of the national data system. 

Linkage among data files. As the different sets of data elements become available for 
operational use, linkages among these sets must be established, tested, and verified. The testing 
and verification also must include assessments of the relative utility of the relational policv analyses 
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generated from the linked data sets. These assessments must continue and encompass the 
increasingly larger number of linked data sets that come on line as additional micro records become 
operational, until the national data system becomes fully operational. 

Concurrent development activities. Although we will not elaborate here, implicit in the 
research, development, testing, and verification activities noted above are such data system design 
and development activities as the identification of essential populations of generalization; the design 
and selection of samples of data suppliers; the design of mechanics for collection of data, including 
specifications for, recruitment of, and training of data collectors; the design of survey field 
procedures; the design of plans for analysis of data and reporting of results; the development of 
systems for transmission of data; and the development of software systems for data receipt, 
control, editing, analysis, and summarization. 

In the following table, we provide a truncated outline which is illustrative of the development 
and phasing activities discussed above. For example, under the category "Environment" we 
identify the subcategory "community and family characteristics" and indicate that, at the present 
time, data in this subcategory are collected only in aggregate form in the data collection activities 
currently being conducted by the Federal Center for Statistics. In Phase I, research and 
development activities would be undertaken, in Phase H, testing and verification would take place, 
and in Phase HI, the data would become available in micro record form. The remainder of Table 1 
can be read in the same fashion; in the subcategory "school goals" for example, research and 
development activities would not begin until Phase HI, whereas in the subcategory 
"dropouts"- where data in aggregate form already are being collected-research and development 
activities would begin immediately in Phase 1. Note that Table 1 illustrates only the first three of 
ten proposed development phases. 

Establishing priorities for development. Table 1 is only illustrative of the types of decisions 
that would have to be made in developing a national data system for elementary and secondary 
education. The actual choices of the order in which categories and subcategories of data elements 
would be developed must be made by the Federal Center for Statistics, the Office of Educational 
Research and Improvement, and tne consortium of local, state, and federal agencies described 
below. However, based on our intensive and careful review of the needs expressed by authors of 
the papers underlying the Synthesis Report we proposed consideration of the following priorities. 

The conceptual model that is defined in Chapter 4 provides categories of data that are 
required to meet the information needs of education policy-makers at several levels of government, 
as well as those of the new constituencies for information identified in other parts of this report. Of 
all data categories defined by that model, school process information is least available now. School 
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Table 1. Illustrative Status of Categories and Subcategories of Data at Various Phases of 
Development of the National Data System 



Categories & 
Subcategories 

Environment: 

Family and commun. 
characteristics 

Resources: 

Financial revenues 

Educative Difficulties: 
Handicapped status 
Motivation 

Educative Goals: 
School goals 

Participation: 

Course enrollment 

Outcomes: 
Dropouts 



Current Phase I 



Ag. 



Ag. 



Non. 



Ag 



Status of Data Category 
Phase II Phase III 



Ag.(R&D) Ag. (T.&V) Micro. 



Ag, 



Ag. (R&D) Ag.(T&V) 



Ag. Ag.(R&D) Ag.(T&V) Micro. 

Non. Non.(R&D) Non.(R&D) Non.(T.&V) 



Non. 



Non. 



Ag.(R&D) Ag.(T&V) 



Non.(R&D) 



Micro. Micro.(R&D) Micro.(T&V) Micro.(Rev.) 



Micro. 



LEGEND: 

Non. 



Ag. 

Micro. 

(R&D) 

cr&v) 

(Rev.) 



denotes a data subcategory that does not presently exist in the set of projects 
operated by the Federal Center for Statistics. 

denotes a data subcategory in which data are presently collected only in aggregate 
form in the set of projects operated by the Federal Center for Statistics. 

denotes a data subcategory in which data are presently collected in the form of 
micro records in the set of projects operated by the Federal Center for Statistics. 

denotes a data subcategory in which research and development is to be conducted. 

denotes a data subcategory in which testing and verification is to be conducted. 

denotes a data subcategory in which the data previously existed in the form 
indicated, but for which revised data elements are developed and adopted. 
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process information includes; information on the educative goals of the schools; on allocated 
resources- facilities, staff, equipment and materials; information on educational pursuits curricular 
offerings, standards, teaching-related and school-related activities; and information on pupil 
participation in the process of schooling. There is also a critical need for high quality outcome data. 
The best outcome data available are presently provided by NAEP. However, these data are limited 
to students at relatively few grade levels, are only collected biannually, and are limited in subject 
mattet tested. Therefore, in our judgment, two categories of data — school process data and 
outcome data — deserve priority in the development of the national data system. However, 
although information on process and outcome have the highest priorities in terms of need, as a 
practical matter, the data system should attempt first to develop micro record on a small subset of 
data to develop the collection process and refine the data base development process. Information 
on pupil participation which would provide data for enrollment and attendance would be the 
priority candidate for initial development. The research effort to develop a more comprehensive set 
of process and outcome data should be given high priority and proceed on a parallel track. 

School context information should constitute a third area of priority development; 
particularly information that describes the environment in which the schools operate, such as 
community and family characteristics and expectations, as well as information that describes the 
educative difficulties of students. In our judgement, these two categories of data should receive 
attenti * n once the development of micro records is well underway in the school process and 
outcomes categories. 

Our fourth order of priority would be to address data needs in the educative goals category. 
A final priority, but certainly essential, would be the categories of incoming and allocated 
resources, including revenues, and expenditures for, and stocks of, materials, equipment, facilities, 
and personnel. As is clear from Table !, above, certain existing aggregates are recommended for 
phasing into micro record formats stages beyond Phase I. This raises the issue of parallel 
aggregate reporting for existing aggregate data series to allow users to move from the old 
problematic series to the micro record based series. This overlap should be carefully planned into 
the phasing of the new system. 

Description and timing of the development and phasing of state involvement in the national 
data system. For the data system to be truly national, states must be involved in each phase of the 
development and maintenance of the system. One of the challenges to the Federal Center will be to 
ensure that this kind of state participation actually takes place. 

The Department of Education has already begun to involve state governments in the 
process. Copies of the Synthesis of Invited Papers, which were commissioned to examine the 
needs for a new national data system for elementary and secondary education, have been sent to 
each of the governors, the leadership and education commiuees of the stage legislatures, the chief 
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state school officers, and the various associations of state entities. Copies of an early draft of this 
document were also made available to staff members of some of the same associations for 
comment. Once the final decision is made to implement a new system, a formal federal/state 
mechanism must be established to plan and monitor the development of the national system. 

A first step will be the establishment of a consortium to provide the planning mechanism. 
This consortium would be made up of representatives of the state governments as well as selected 
representatives of private school groups, persons from academia, representatives from the 
Department of Education and others interested in education. The consortium would be established 
through invitations sent by the department to each state. The governors, legislative leaders and the 
chief state school officers of each would be involved in the selection of a representative of that state 
to the consortium. At the same time the Department would nominate members who would be able 
to represent the views of other data providers and data users. 

It is expected that the consortium will be appointed in the first quarter of 1986, and could 
have its organizing meeting in April. In preparation for the consortium's first meeting, the Center 
would prepare a draft agenda which would be circulated to the membership for comments. 

The first major task of the consortium would be to select a committee to consider specific 
information requirements and recommend development and phasing priorities, with special 
attention to the standardization of data definitions for the system. This activity would be timed to 
coincide with and become part of the development and phasing effort described above. The 
Committee would consist cf persons selected by the consortium as well as appropriate ex officio 
members of the staff of the Center for Statistics and other parts of the Department of Education. 
The committee as well as the consortium would be staffed by the Center. 

There are several states which are now seriously considering the development of integrated 
management information systems. Another committee could be established, composed of the 
appropriate technical personnel from some or all of those states, to review the progress being made 
by each, and to attempt to develop common features which could then provide data to the Center 
under the provisions of Alternative B. Depending on the number of states involved, a limited 
number of "observer" states could participate in this effort 

The consortium would also consider other issues and establish such committees as are 
required to carry out its work, within the limits of the Center's ability to provide support. 

It is expected that the full consortium would meet no more frequently than once a year. 
However, the committees would meet on an as needed basis and prepare reports which would be 
reviewed and approved by the membership and published as technical assistance for others. 
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